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SUMMARY 

4 

Population model coefficients were chosen to simulate a saturated 2 fixed- 
effects experiment having an unfavorable distribution of relative values. Using 
random number studies, deletion strategies were compared that were based on the 
F-distribution, on an order statistics distribution of Cochran's, and on a com- 
bination of the two. The strategies were compared under the criterion of minim- 
izing the maximum prediction error, wherever it occurred, among the two-level 
factorial points. The strategies were evaluated for each of the conditions of 
0, 1, 2, 3, 4, 5, or 6 center points. 

Three classes of strategics were identified .as being appropriate, depending 
on the extent of the experimenter's prior knowledge. In almost every case the 
best strategy was found to be unique according to the number of center points. 
Among the three classes of strategies, a security regret class of strategy was 
demonstrated as being widely useful in that over a range of coefficients of vari- 
ation from 4 to 65 percent, the maximum predictive error was never increased by 
more than 12 percent over what it would have been, if the best strategy had been 
used for the particular coefficient of variation. 

The relative efficiency of the experiment, when using the security regret 
strategy, was examined as a function of the number of center points, and wjs found 
to be best when the design used one center point. 


INTRODUCTION 

The two-level, fixed effects, full or fractional-factorial designs of exper- 
iment, without replication, are often appropriate for those situations where the 
experiment is very expensive or time consuming. An example of costly experiment- 
ing is provided by the destructive testing of simulated aircraft turbine engine 
components, as in the rotor burst protection testing described by Mangano (1977). 
A rotor burst protection investigation was planned as a two-level fractional 
factorial experiment to measure- the containment efficiencies of composite struc- 
tures. The description (Holms 1977a) of that experiment is illustrative of one 
area of applicability of the results of the present Investigation. 

If a two-level full- or fractional-factorial experiment is performed and 
n, observations are obtained from n^. orthogonal experimentiil conditions, the 
appropriate empirical equation for representing the results can have as many as 
n^. terms, each with a coefficient t!iat has been fitted to the data. When this 
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Is done, a question that should be asked is: "Can the predictive accuracy be 

improved If some of the terms are deleted?" The fact that some of the terms 
might degrade the predictive accuracy of a fitted equation was recognized by 
Walls and Weeks (1969) but they gave no procedure for identifying such terms. 

A method for the sequential deletion of terms that was Intended to reduce 
the prediction error was given by Kennedy and Bancroft (1971). Their method 
assumed that the experimenter has a prior established order for subjecting the 
terms to a sequence of significance tests. Unfortunately, in many experimental 
situations, there Is no subject matter basis for establishing a prior order, 
and in such cases an order statistics procedure is appropriate. An order sta- 
tistics approach for significance testing was used in a pair of related papers 
by Daniel (1959) and by Blrnbaum (1959). They were not then seeking to minimize 
prediction errors. 

For model selection procedures used with small saturated experiments (exper- 
iments designed to have only as many experimental conditions as there are model 
parameters to be fitted), the analysis should begin with a minimum number of 
estimable terms being sacrificed to form a denominator for the test statistic. 

A procedure using m-terms sacrificed, where m can be as small as one, was 
investigated by Holms and Berrettoni (1969). The object was to delete terms 
in a manner where some control was maintained over the probabllltie5 of Type 1 
or Type 2 decision errors. 

The minimizing of prediction error was the object of an investigation of a 
chain pooling strategy as described by Holms (1974). Whereas that investigation 
had assumed that only one cycle of analysis would be used, a suggestion given by 
Holms and Berrettoni (1969) was that more than one cycle should be used. The 
purpose of a further chain pooling investigation (Holms (1977b)) was to optimize 
a combined procedure that might contain more than one analysis cycle, where the 
procedure is to be optimized for minimum prediction error. An Important applica- 
tion of chain pooling occurs in empirical optimum seeking. 

A widely accepted methodology for the design and analysis of experiments 
that are efficient for the empirical attainment of optimum conditions was intro- 
duced by Box and Wilson (1951) and refined by Box and Hunter (1957). These 
methods are now known as response surface methodology. Designs that are optimal 
for fitting second degree equations were studied by Lucas (1974 and 1976), who 
was concerned with the optimality of single block designs, but multi-block 
designs are often appropriate in the applications of response surface methods. 

A catalog of multi-block designs, limited to those particularly applicable to 
response surface mothixls, was given by Holms (1967). Response surface method- 
ology assumes that liypercube and star blocks will contain "center points," 
Criteria for the numbers of such points to use, together with tables of recom- 
mended numbers, were given by Box and Hunter (1957), Criteria leading to much 
smaller numbers of center points for single block experiments were given by 
Lucas (1976). The purpose of Holms (1979) was to characterize tlie experiment 
designer's options for numbers of center points in a range from very small to 
moderately large for multiblock sequential designs. The multiblock sequential 
designs were tliose for which treatment tables had been given by Holms (1967). 

The numbers of center points used in eacli of the hypercube blocks ranged from 
zero to six. 
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The purpose of the present Investigation Is to identify chain pooling 
type sequential deletion procedures that will minimize tlie prediction errors 
in models fitted to the results of experiment designs having 16 hypercube 
points and any of zero to six center points. The investigation used Monte 
Carlo studies, and the results are exhibited as tables giving some of the 
operating characteristics of admissible strategies for each of the center 
point options. A security regret strategy is identified within each set of 
admissible strategies, and it is shown to be useful for a wide range of coef- 
ficients of variation. 


MULTISTAGE DECISION PROCEDURE 
Population Model 

The single observation value of the response is assumed to occur accord- 
ing to the model 


y = E(Y) + e (1) 

where e is independently normally distributed with mean zero and variance 
0 ^. (Some robustness against nonnormality for a chain pooling procedure was 
demonstrated by Holms and Berrettoni (1967).) 

For relatively saturated experiments that are smaller than 16 observations, 
the opinion is offered that such experiments are too small to provide both 
(1) good estimates of model coefficients and (2) a good test statistic, in 
cases where random errors are large enough to call for a statistical decision 
procedure. The simulations of the present investigation were all performed 
with experiments containing 16 hypercube points plus zero to six center points 
in the belief that such experiments are large enough to justify the use of a 
statistical decision procedure, but small enough so that the precise optimiza- 
tion of the procedure would be quite beneficial. Where g is the number of 
independent variables, and the experiment is assumed to be a 2"*^ fractional 
replicate of the full factorial experiment, the factorial observations are 
assumed to result one-for-one from the hypercube points and their numer is 

Op = 2^"^^ = 16 

An exiimple of an appropriate mtxiel equation for the population mean value 
of the response in the case of four independent variables is 

E(Y) = + B2XJ + B3X2 + t^4Xj^X2 + B5X3 + B^XjXj + 67X2X3 


+ 6gXjX2X3 + 6qx^ + 6 jqXjX^ + ‘^ll’'2’‘4 ^12’‘l’^2^4 


+ 6,,x„x, + 6,,x,x^x, + ^x x„x, + 6 x x x x 
13 14 14 134 15 234 16 1234 


( 2 ) 


The model is assumed linear with or thogonallty of parameter estimates provided 
by tlie design of the experiment or by an ortliogonal izlng transformation (Holms 
(1974)). The subsequent discussion assumes tluit an equation such as (2) will 
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bp fitted to the results of a two-level experiment where the x's are ’’design 
values,” namely, the high level of xj^ is represented by xj; ■ +1 and the 
low level of Xj^ is represented by x]^ “ -1. (If center points are used, 
they have coordinates with all xj^ " 0) . 

The initial model fitting is assumed to give least squares estimates of 
the model parameters that are minimum variance unbiassed linear estimates and 
for parameters beyond 0]^ have the form 

k“l 

where 1 = 2, . . . , n^ and the a^i^ are appropriate values of plus or minus 
one. Such estimates have expectations 

E(bi) * 0j (4) 


Combination Estimate For Zero Degree Coefficient 

A weighted estlmrte of the of equation (2) is to be formed from the 

nc hypercube observations and the ng center point observations where all 
observations are assumed to have variance o2. Model coefficients estimated 
from the hypercube observations each have variance 

V(bi) * o^/nc 

Thus, the variance of the function estimate for a model such as equation (2) 
with coefficients all estimated, for example, by Yates' method from such 
observations is (at the design center) 

V(Yq) = V(bj) = o2/nc 

Let Yg be estimated from a combination of the n^. hyper cube points and the 
ng center points. Let yg be the arithmetic mean of the ng center point 
observations. Then the estimate of Yg weighted inversely as the variances 
of b^ and yg is 


Yg = (n^b^ + ngyg)/(n^ -*■ ng) 

Because the coefficient estimates are uncorrelated, the weighted estimate Yg 
is also the least squares estimate of Bj. Thus, if bj is the estimate of 
the zero degree coefficient from the Yates analysis, the least squares estimate 
from the combined observations is 

b* = (n^b^ + Vg)/(\ 


(5) 
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Mean Squares and Sums of Squares 

The squares of the estimates multiplied by n^ provide the numerator mean 
squares used in the hypothesis testing. 


Zi - n^bi 

These mean squares have expectations 

E(Zi> = o2 + 


where 


for 


V(Y^) = o" 


i “ 2| •••> Of 


( 6 ) 


(7) 


( 8 ) 


and for 


k “ 1, .... Oj, 

Thus, from equation (7) if any Bj is zero, the associated Zj is an estim- 
ator of o2. 

The denominator for the hypothesis testing is based on the construction 
of sums of squares. Six cases are identified according to combinations of the 
values of oq and mp where ng i« determined by the design of the experi- 
ment and mp is chosen according to a strategy of hypothesis testing. The 
cases are identified by the first three columns of table 1. Equations for the 
initial sum of squares, SSg, to be used in the starting denominator of the test 
statistic are derived in appendix B and given in the fifth colunm of table 1. 


Sequential Deletion 

Because case a provides no denominator sum of squares, there can be no 
deletion procedure. All the model coefficients are estimated and all the terms 
are retained. 

Case d uses ng * 0 and mp > 0. This is the case investigated by Holms 
(1969). The deletion method of that investigation was as follows; 

The mean quares, Zj, from the usual Yates' analysis (aside from the zero 
degree coefficient) are ordered in nondecreasing magnitude and renamed Z(j): 

Z(i) < ••• s Z(n) 

where j = 1 n and n * n^ - 1 • As stated by Birnbaum (1959) the 

optimal decision procedure wlien all except possibly one of the coefficients of 
an equation such as (2) are zero uses a test developed by Cochran (1941). The 
statistic is: 


Cn = Z(n)/(Z(1) +••••+ Z(n)) 
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Chain-pool Ing assumes thai mp of the smallest ^‘(J) been generated 

with zero population coefficients. Tlielr sum is called Rj-l vrtiere initially 

J - 1 ■ mp 

Ifeiltlpllcatlon of the critical points of Cochran's distribution by J gives 
the critical points of the Uj distribution tabulated by Holms and Berrettoni 
(1967). The mean square Z(j) is tested for significance at nominal prelim- 
inary level Qp using the statistic 

Uj - JZ(j)/(Rj-i Z(J>) <9) 

If Z(j) is not significant, J is indexed upward by one and the next mean 
square is tested. If any mean square so tested is significant, (e.g., the 
j-th) then each subsequent larger mean square is tested at a nominal final 
level Of, where aj 4 op. For example, the k-th mean square is tested using 
the statistic 


Uj - jZk/<Rj-i Zr) 

If the k-th ordered mean square is the smallest mean square to test signifi- 
cant at level Uf then all terms associated with smaller mean squares are 
deleted from the model. Because the assumptions of Cochran's distribution are 
thereby repeatedly violated, the useful values of the strategy parameters 
(mp. Op, Of) were determined from simulation studies. 

The generalization of the strategy (mp, Op, af) investigated by Holms 
(1977b) included a sequence of analysis cycles, but showed that merely one 
cycle was sufficient. The cases with no ^ 0 are cases b, c, e, and f. 

A hypothesis testing procedure more general than Holms (1977b) is appropriate 
for these cases. 

Consider cases b and c where np ^ 0 and rap “ 0. The first mean 
square to be tested is Zq) and the null hypothesis is 

Hot P(i) - 0 

where for any j, is the parameter associated with the ordered mean square 

7.Qj. The alternative hypothesis is 

H^t 3(1) > 0 

Because the Uj is not defined for j 2, Z(\) cannot be tested against 
the Uj distribution. If the test is performed against the F-distrlbution, 
the fact that Z(j) is an ordered statistic implies that a test of nominal size 
a will not have true size a. With this proviso, a nominal size a-test is 
performed . 


For case b the test statistic is 


l,ndfb ss 


"dfbfjQJt 


( 11 ) 
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with tijfj, defined by equation (R3) iind wltli SS|, computed by equation (BA). 
For case c the equiv.alent test statistic is 


IT =. 

*^l,ndfc sSc 

where njfc Is given by equation (B6) and SSj. is given by equation (B5). 

If Z(x) is reported significant then no further testing is done and 
there is no conditional deletion of terms. 


( 12 ) 


For either of cases b or c let SSq be the initial sum of squares and 
let OjifO *56 i* 5 e initial degrees of freedom. If was reported as insig- 
nificant, then it is pooled with SSq for a test of Z^2)* test statistic 

is then 


^I,ndf0+1 


^"dfO ^^^(2) 

SSq + 


(13) 


Testing and pooling continue in this manner, provided in-Jignif icance is the 
result of the prior test. 


The test statistic for any is thus 


^1 -f(Hj 


<"dfO * 

SSq + Z(1, + •••• + Z(J) 


(14) 


For J ^ 1 the option exists of testing Zq) against the F-distr ibution 
or against the Uj-dlstr ibution. These options are also both available for the 
first test of a Z(j) in cases (d), (e) , and (f), however, testing against the 
F-distribution might not be good for case (d), because in case (d), there is 
neither a pure error nor a residual sum of squares, and the testing is performed 
entirely with ordered mean squares. 


Suppose the situation is that of oq > 0 .and j > 1. A criterion is 
needed for choosing between testing against the F-dlstributlon or against the 
Ih- distribution. If j Is relatively small and no is relatively large, the 
F-distrlbutlon might be more appropriate, whereas if J is relatively large 
and no is relatively small, the Uj-ii istr ibution might be more appropriate. 
One approach could be, for j '■ 1, to compute j/nQ and use the F-distr ibution 
for j/no i rp and use the Uj-distr Ibution for J/uq > fp where 0 ^ rp • 
and where rp has been optimized from Monte Carlo studies. Table 2 shows how 
the choice of the value of rp affects the values of no and j at which a 
transfer occurs from the use of the F-dlstributlon to the use of the Uj-distri- 
butlon. 


Consider the case 


.)/no " ''F 


(15) 


Tlae Uj statistic is defined by equation (9). 


Suppose the criterion j > rpnQ 
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has been met and the information in ygi* ***< yOnO combined with 
that in Zqj for a test of Z^\ against a critical point of the 
Uj-distribution. An approximation to equation (9) is 


<»dfO 

“ SSo + 2 ^) + + Z(j) 


(16) 


The distribution of tj of equation (16) is merely an approximation to 
the distribution of Ui Oi equation (9) because the denominator of equa- 
tion (16) has been stabilized by the n^fQ mean squares in SSq. 

Under the null hypothesis, z(j) Is the largest of an ordered sample of j 
estimators of o2. Although the quantity (SSq + 2(1) + **• + 2(j))/(ndf0 J) 
is an estimator of o2 as is the quantity (^j-i + equation (9) 

the quantity J of equation (16) rather than the quantity (n^jfo + j) was used 
as the entry point of the Uj -tables, because the tables are based in part on 
the numerator z(j) being the J-th extreme value of a sample of mean squares 
together having mean value o2. 

For case d . use of the Uj-distrlbution implies that the first test using 
equation (9) takes place with 


*■)-! “ ®^d 

where SS^ is defined by equation (B7) and j - mp + 1. Subsequent testing 
is done with 

^ -*^ (J) 

For either of cases e or f . with j i rpng. the test statistic is provided 
by equation (14) with SSq and Ojjfo being given by equations (B9) and (BIO) 
for case e and by (Bll) and (B12) for case f. If Z(j) tests as insignifi- 
cant, then it is pooled with (added to) the denominator of the test statistic 
and j is indexed upward one unit. When some Z(j) tests as significant, 
the testing is stopped and n terms are deleted iron the model where n is 
the integer value of r^(j - 1) where r^ has been empirically optimized. 

The terms deleted are those corresponding to the n smallest mean squares. 

If ^(i) tests as insignificant at J “ n - 1, then n is the integer value 
of r,,tnc - 1). 


Definition of Strategy 

In sunanary, the expressions for degrees of freedom and sums of squares are 
given in table 1, and the enl ire sequential deletion strategy is specified by 
the parameters (rap, rp, ap, ay, r^). Where mp has range 0 ^ mp < nc« «p is 
the number of moan squares initially pooled , if ^(j) 1® -he j-tn ordered mean 
square being tested, then Zq) is tested against the U.- distribution at nom- 
i nal le vel ay if j “• 1 and J > rpng. The mean square is otherwise 

tested against the F-distrlbutlon at nominal level ap. The convention 
Op “ l.O is used to signify that no testing is done against the F-dist;‘ibution, 
and the convention ay • 1.0 is used to signify that no testing is done against 
the Uj-dlstribut ion. The number of terms found to be Insignificant is 
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multiplied by rr, and the integer value of the product . namely» n, la the 
number of amalleat abaolute value coefficicnta whose terms will be deleted 
from the model . (The coefficient, b*. of the zero degree term is excluded 
from the deletion procedure.) In the notation for the strategy parmeter 
set (mn, rp, ap* a long dash will be us^ to represent a parameter 

if it nas been made Inoperative by a value assigned to scmie other parwaeter. 


SIMULATION PROCEWRE 
Unfavorable Population Model 

The basic chain pooling concept was investigated for the purpose of min- 
imizing prediction error as described by Holms (1974). That investigation 
was concerned with the fitting of models to fractional factorial experiments 
under the condition of population functions of irregular shape. The present 
emphasis is on the condition where the relative values of the population 
coefficients are all unfavorable to the deletion procedure. For reasons 
given by Holms and Berrettoni (1969) this condition is achieved by proportion- 
ing the squares of the values to the expected values of the order statis- 
tics of a single distribution. 

To accomplish such proportioning, let Ck fhe expectation of the k-th 
order statistic among p itwlependent statistics. Let 


6^^-Wl k-l.*--.p 

Expectations of order statistics from a gamma distribution with scale 
parameter one. shape parmeter 1/2 and many sample sizes have been tabulated 
(Harter, H. L. (1964)). Multiplying such values by 2 gives the expectations 
of the order statistics of the central x|j^^ distribution. Such expectations, 

for a sample of size p, provide the values called for by the definition of 
the 


Equation 

dents: 


(13) of Holms and 


®i 


Berrettoni (1969) 
1/2 


now gives for the coeffl- 


(?>) 


(17) 


where k« 1, 2, 3, •••, p; i ■ 1, 2, •••, n^ and i ■ k + 1 for k ■ 1 , * • * , 
p: aiui 6^ ■ 0 otherwise. 


The statistician's strategy consists of the conponents (mp, rp, ap, ay, 
r^). Nature's strategy consists of the msnber, p, of non-null population 
parameters, and the mean noncentrality narameter, X. Let 


From equations (17) and (18) 


0 ^ I. — 

28 “h 


(18) 


■ 6o6jj 


(19) 


in 


Because of the freedom to choose the value X and because X Is a scale , 
"y ^ 

parmneter on o- (eq. (17)) an invest Inatlon of the effect of variations in a 

is superfluous, and will be sot equal to one. 

In general, the smaller the number of null mean squares, n» the greater 
will be the probability of decision errors. This was illustrated in figure 4 
of the paper by Holms and Berrettoni (1969). Thus, the most difficult situa- 
tion arises for n ■ 0. For the 2 B"h experiment with g - h • 4, and where 
the 6i term (zero degree term) is not subjected to testing, the condition 
equivalent to n ■ 0 is the condition p ■ 15, 

As developed by Holms (1977b), a nature's strategy with p 15 and a 
normal distribution of model p.iramoters would seem to be a 1 ghly likely 
strategy, and correspondingly, a statistician's strategy optimized against 
such a nature's strategy should be thought of as a Bayes strategy . As devel- 
oped by Holms and Berrettoni (1969), such a normal distribution of model 
parameters is represented by the parameter distributions of table 3 and these 
distributions are highly unfavorable to the statistical decision procedvire. A 
proc^ure optimized against p ■ 15 and the distribution of of table 3 

may therefore also be regarded as a security strategy . Such a nature's stra- 
tegy (table 3) will therefore be chosen as the strategy against which the 
statistician's strategy will be optimized, and such optimization will there- 
fore combine the Bayes and security attributes. 


Steps of Simulations 

Ordinarily, in the analysis of a real experiment, X’ates' method would be 
applied to the observations to give estimates of the model coefficients. The 
population mean values and the errors of the observations would be unknown, 
but in this investigation, the values of the population mean values are re- 
quired to be known. The steps in a simulated experiment were as follows: 

1. An unfavorable set of were constructed as Indicated by 

equation (19). 

2. Population mean values, for the simulated observations were 

computed from the using tlie reversed Yates' method of Duckworth (1965). 

3. Pseudo normal random errors, ej, were generated as described by 
Holms (1977b). 

4. Tile simulated observations, Vj, were generated by 

yj = m + ej (20) 

5. The bj were estimated from the vj using Yates' method, except 
for b*. whlclj weighted in the Vqj^ as in the preceding section (eq. (5)). 

Some of tl\e hj were set equal to zero using the strategy 

(mp. r^., Up. iy, r,^) 
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7. The reversed Yates* method o£ Duckworth (1965) was used to compute 
predicted values, y^, using the reduced set of b^. 

8. The prediction errors were computed from 

®pi “ yi “ >^i 


( 21 ) 


1,2, •••,16 


Additional details of these steps are given in appendices C and D. 


Magnitude of Scale (Noncentrality) Parameter 

Any particular strategy (mp, ry, ay, ay, r^) was evaluated for an array 
of populations having five unique values of the mean noncentrality parameter, 
X, namely, 0.25, 1.00, 4.00, 16.00, and 64.00. From equation (18) with 
n^. = 28 ““ = 16 the corresponding values of 6 are 0.125, 0.25, 0.5, 1.0, 
and 2.0. 

As developed by Holms (1977b), the reduction of V(Yi) achievable by 
deleting terms is a^/nc for each term deleted. On the other hand, if equa- 
tion (2) is the population model, and if the x-values are all +1, the bias 
in Y is increased by the amount of 6i for each Bi value that is deleted. 
Thus, an optimal strategy to minimize the squared error of ^ should not only 
delete all terms for which the population is zero, it should also delete 

at least all terms for which the bias contribution to mean square error is 
less than the variance contribution. 

Because in the simulations all Si i 0 and as Indicated by equation (2) , 
for that point of the experiment where all of the x-values have the value +1, 
the '"'T)ected value of Yi takes on its greatest absolute value, which is 
(eq. (19)) 


max 

%ax = i nE(yil] 


P 

j'l 


p 

The values of ^ 06 for p = 15 have been listed in table 3. Because 
3=1 j 

0 = 1 , these values are also the vajjes of Pmax^®’ 

Reciprocals of Pmax/® here defined as coefficients of variation 

for the maximum population mean values. From table 3 such coefficients vary 
from a high of 64.3 percent (at 0 = 0.125) to a low of 4.0 percent (at 0 = 
2.000). This range of such a coefficient of variation suggests that the range 
of 0.125 i 0 ^ 2.000 is an adequately wide range of 0 to represent the situ- 
ations that an experimenter might encounter. 
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EVALUATION CRITERIA 


Where ©oin "observation" errors, namely, the pseudo normal 

random numbers generated in the nth simulation, the "observations" are given 
consistently with equation (20) by 


^Olln " *^i/ ^ ®0ln 


( 22 ) 


( •• 1 ,2 , . . . , I qI I “ 1,2, ...,n^ 

Following the selection of terms (vrticrc some of the coefficient esti- 
mates are set equal to zero), the predicted values of the dependent variable 
are computed for all the hypercube combinations of the Independent variables, 
by the reversed Yates' method of Duckworth (1965). The difference between 
predicted values, ypt^mn* dependent variable for the nth simulation 

and the population mean will be called the prediction error, and thus it 
is (consistent with eq . (21)): 

®pi tmn " VplEran ~ ^ii 

£ “ l,2,...,£0i m ~ 1 , 2 a • . • i * l, 2 ,a..,n^ 

Over the ng simulations, the sample mean square error of prediction for 
a given treatment is 


ne 

I V 2 

^ 4? 


1^2 „ J, 

*^pifm no '^pi£mn 
n=l 


(24) 


The maximum of such errors over the treatments is 


2 _2 max 

max p£m,max 1 = 




(25) 


The mean of the squared error over the simulations and over the points of 
the space of the experiment is 



nc 

S. £ 

"c w 



(26) 


Equations (25) and (26) provide two criteria for measuring the effective- 
ness of a strategy. The particular sot of values of strategy parameters that 
minimizes Cptm max given by eq . (25)) cun be called a security strategy, 
and if the points of the space of the experiment are assumed to be equally 
likely of being of interest, the particular set that minimizes ^pjm 
called an approximate Bayes strategy. For cither criterion, the values of 
squared errors would liave been the prime consideration. 
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The criteria of equations (25) and (26) were evaluated using computer 
simulations using 1000 experiments. Thus, the long run mean squared error 
of the decision procedures was evaluated. This leaves open the question cf 
how badly a decision procedure might perform in individual cases. One 
approach to this question is to evaluate the stability of the mean squared 
errors observed in the simulations. Thus, in addition to the criteria of 
. nations (25) and (26) two other criteria for the effectiveness of a strategy 
\ :e investigated. They are concerned with the stability of the quantities 
defined by equations (25) and (26). The instability of these criteria can be 
measured by the variance of the square of the prediction error. The estimate 
of the variance of is 



Equation (27) gives an unbiased estimate of the variance of the squared error 
over Hg simulations. The maximum of this quantity over the space of the 
simulated experiments is defined by 

, . . . ,Og 

The arithmetic mean of the variance of the squared error over the space 
of the experiments is defined by 

"c 

1=1 


The average number of terms, selected by the strategy, is computed 

for each of the values of Oj, £=1, ...,to and for each of the values of 
nQ, m = l,...,nn0* The program also computes the ratio of the maximum predic- 
tion error to the scale parameter 0. The ratio is computed from Ojj, and 
from the of equation (25): 


Ce 


e ,mx 


( 0 ) 


'pirn, max 

0e 


(30) 


The value of Cgg^,^x(0) of the preceding equation was adjusted to penalize 
it for the increased experimentation needed for the reduction in variance that 
might be expected from the additional center point observations. Thus, with 

"t "c + "0 


^'ae , mx ^ 0 ^ 


n 


l/2p 

t ee,mx 


( 0 ) 


(31) 


u 


Although the experiments were simulated, the model fitting and selec- 
tion was performed, and predicted values were computed as if the experiments 
were full factorial experiments, the conclusions ot the investigation are 
not necessarily limited to full factorial experiments. The errors of the 
predicted values were always evaluated at points of the space of the experi- 
ment for which "observations" were available. Thus, the conclusions of the 
experiment are equally applicable to regular fractional factorial experiments 
with 16 treatments, provided that the only concern is with prediction errors 
at the points of the experiment where observations were actually acquired. 
Thus, for example, if the experiment were a one quarter replicate on 6 inde- 
pendent variables, the strategy recommendations apply to predictions for the 
16 hypercube conditions actually performed. The errors might be much larger, 
and a different sequential deletion strategy might be preferred, if predic- 
tions were to be made for some of the 48 treatments that had not been per- 
formed. As shown by Holms (1974), such predictions should be based on a far 
more stringent deletion strategy than for the case of predictions limited to 
points of actual observations. 


COMPUTER PROGRAM 

Computations were performed using the computer program, P00L9U. Details 
of the program are given in appendices C, D, and E. The manner of repeated 
use of arrays for the simulated observations and estimated model parameters is 
shown by figure 1. The major program logic is exhibited by figure 2. The 
branch points for the computation of sums of squares according to the six cases 
of table 1 are exhibited by figure 2(a). The procedure for the significance 
tests is exhibited by figure 2(b), and the final deletion procedure is exhib- 
ited by figure 2(c). 


SIMUUTION RESULTS 

Results of an investigation of the effect of Monte Carlo sample size on 
the stability of the empirical results for a similar chain pooling strategy 
were given by Holms (1977b). In general, the results converged to a constant 
when the number of sampled experiments was 1000 or more. Variability of re- 
sults occurred as the number of sampled experiments was reduced below 1000. 
All of the strategy comparisons of the present investigation were performed 
for 1000 sampled experiments. All simulations were performed for p * 15. 

The strategies were compared in terms of the maximum coefficient of error, 

Cjjg mx* adjusted for nQ» as defined at equation (31). 


Large Coefficient of Variation 

The investigation of Holms (1977b) concerned the case of ng ” 0. One of 
the conclusions was that if the Investigator has prior knowledge that the rela- 
tive error is quite large (coefficients of variation in the neighborhood of 
65 percent), tl»e strategy should immediately delete the five smallest absolute 
value terms and then test with continued pooling at a nominal test level of 0.05 
against the Uj-distribution, to estimate a number n of insignificant terms. 
The optimum number of terms deleted from the model was shown to be the integer 
value of 1 + 0.675 n* 
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The strategy parameters of the present Investigation are (mp, rp, ap, 
au, and r^). With no = 0, no testing or deletion can be accomplished unless 
mp > 0. As exhibited by figure 2(b), the initial value of j is mp + 1. 
Thus, even if ap < 1.0, figure 2(b) shows that with rap ^ 0, J is > 1, and 
with ng - 0, j is ■' rpng for any finite rp, and thus control is transfer- 
red to statement 418, and testing against the F-distribution is excluded. 

Thus, any testing with ng * 0 is done against the Uj -distribution. 

In the present investigation, the number of terms deleted from the model 
is r,^n. In Holms (1977b) the nun.ber was 1 + r 2 n. Thus, for an equal number 
of terms to be deleted in the two investigations, 

r,^ii = (1 + r2>n 


from which 


r„ = rj + 1/n 

Thus, whereas r 2 = 0.675 was found to be optimum for large coefficients of 
variation (0 = 0.125) in Holms (1977b), a value of r^ somewhat larger than 
0.675 should be anticipated to be optimum for ng = 0 and 0 “ 0.125 in the 
present investigation. 

From the preceding discussion, an optimum strategy for ng ■ 0 and 
0 » 0.125 should be anticipated to occur in the domain exteixilng to larger 

values of r,^ beginning with the strategy (nip, rp, ap, Oy, r^) » (5, , 

1.00, 0.05, 0.675). (ap = 1.00 makes rp inoperative in the preceding 
discussion.) This anticipation was confirmed in that the best strategy for 
ng = 0 and 0 =» 0.125 was (5, , 1.0, 0.05, 0.75). 

The best strategies for O = 0.125 and for each of ng ■= 0, 1, 2, 3, 4, 
5, and 6 are listed in the last row of table 4 for each value of ng. 


Small Coefficient of Variation 

If the statistician's loss function is the maximum adjusted relative error 
over the space of the experiment, C.jj, then the strategy that is optimal for 
0 * 0.125 is a security strategy because within the present Investigation 
Cae,mx larger for 6 = 0.125 thiin for any other value of 0 investigated. 

On tlie other hand, if the statistician's loss function is simply the absolute 
value of the maximum squared error over the space of the experiment, namely, 

®max defined by equation (25) then (with any sequential deletion) that quan- 
tity is a maximum within the present investigation at O = 2.000 and thus the 
security strategy for such a loss function would be the strategy that minimizes 
Cae.max (2.000).' 

The strategy anticipated to minimize Cae,mx (2*0) is the strategy with 
no deletion, which is symbolized as (mp, rp, up, Oy, r^) = (0, — — , 1.0, 1.0, 
0.0), which results in ~ * 15. This anticipation was realized for ng “ 0, 
but for the larger values of ng, some deletion (resulting in o < 15.0) 
actually gave the best strategies for 0 = 2.0. (Results and operating char- 
acteristics of the strategies that gave the smallest observed values of 
(2.0) are listed in the first row of table 4, for each value of ng.) 


t 
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Admissible Strategies 

For the purposes of the present Investigation, a strategy will be classed 
cither as admissible or as dominated according to its values of mx^®^ 

both 0 = 0.125 and O = 2.000. A strategy will be said to be dominated if 
for 0 ” 0.125 there is another strategy with the same or lesser 
(0.125) and with a lesser Cae,mx (2.000). A strategy will also be said to be 
dominated if there is another strategy with the same or lesser C^g (2.000) 
and with a lesser Cae,mx (0.125). 

Any strategy that is not dcxninated is defined as being admissible . The 
strategies found to be admissible are listed in table 4, together with some of 
their operating characteristics. 


Security Regret Strategies 

The strategies of table 4 have been listed for each value of ng in the 
nondecreasing order of Cae,mx (2.0). Thus, the first strategy listed for 
each no is the strategy giving the smallest value of Cae,mx (2.0) for the 
given ng. The last strategy listed in table 4 for any given ng is a strat- 
egy giving the smallest value of (0.125). 

The regret function of a statistical decision procedure, as a function of 
a parameter 0, is here defined as the excess loss occurring with the procedure 
at a particular value of 0 as compared with the loss that would have occurred 
had the best statistical decision procedure been used for that particular value 
of 0. For the purposes of the present investigation a regret function R(0) 
is defined for 6 = 0.125 as being the (0.125) for any strategy 

divided by the value of C^g ^ix best strategy for that value of 0, 

and R(0) is defined for 0 = 2.000 as being the Cgg^nix (2.000) for any 
strategy divided by the value of C^g for the best strategy for that 
value of 0. 

Thus, for the successive values of ng, the regret functions R(ng, 0) 

are 


R(ng, 0.125) = Cjjg^ngj(ng, 0. 125) /min[Cag (ng, 0.125)j 

and 


R(ng, 2.0) ^ae,mx^”0’ 2.0)J 

From table 4, the values of 

min^ge,nix^*'0* 0*125)^ and ®^'^[pae,mx^*'0> 2.0)J 


are as follows. 
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The single strategy that has the smallest regret function over both 
0 * 0.125 and 0 « 2.0 is defined as the security regret strategy . The 
security regret strategy is thus the sequential deletion procedure, which 
for a given ng, produces the least increase in prediction error for p » 15 
and an unfavorable distribution of parameters over that prediction error which 
could have been achieved if the best strategy had been chosen for the given 
(unknown) value of error variance, o^. 

In examining the R(0) values of table A for a given value of Oq, the 
parameters that give the security regret strategies are those that give the 
joint minimums on R(0.125) and R(2.0), and these joint minimums have been 
identified by asterisks. Thus, for the given values of ng, the security 
regret strategies and the associated values of Cgg^j„j^(0) are as follows. 


"0 

mp 


«F 


% 

^ae 

,mx (0-125) 

Oae.rox (2*0) 

0 

1 

— _ 

1.0 

0.50 

0.25 


32.95 

2.2AO 

1 

0 

3.0 

.50 

.10 

.80 


31.78 

2.180 

2 

0 

3.0 

.25 

.50 

.85 


33. lA 

2.252 

3 

0 

0.0 

.75 

.50 

.80 


33.29 

2.301 

A 

0 

0.5 

.50 

.50 

.80 


33.89 

2.309 

5 

0 

1.0 

.25 

.10 

.80 


33.97 

2.39A 

6 

0 

0.5 

.50 

.05 

.80 


3A.72 

2.A08 

The 

question can 

be asked 

as to 

what 

choice 

of ng will 

result in the 


most efficient experiment. If the object of a choice of ng is to use the 
most efficient choice together with a security regret strategy for deleting 
terms, then the preceding table shows that the most efficient choice (the 
choice that minimizes each of (0.125) and Cae,mx (2.0)) is the choice 

of ng = 1. This choice applies to the condition of n^, = 16. 


"0 

"'l"[fae,mxS' °-^25)J 

"‘l"[^ae,mx<"( 

0 

29. A6 

2.065 

1 

30.39 

2.118 

2 

31.11 

2.187 

3 

31.95 

2.2AA 

A 

32.63 

2.300 

5 

33.51 

2.352 

6 

3A.25 

2.A01 


Selection of a Strategy 

In summary, if the experimenter wishes to minimize the maximum prediction 
error over the 16 hypercube points of an experiment with ng center points when 
the variance error is relatively largo (coefficient of variation in the range of 
65 percent), the strategy for a given ng should be the last listed strategy 
(for the given ng) of table A. If the experimenter wishes to minimize the 
maximum prediction error over the points of the experiment when the variance 
error is relatively small (coefficient of variation in the range of A percent) 
the strategy for a given ng should be the first listed strategy (for the 
given ng) of table A. 
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If the experimenter has no basis for a choice of one of the two preceding 
extreme choices, the choice should be a security regret strategy as Indicated 
by the asterisked results in table 4, In which case (for all of the ng 
values) the largest value of the regret function will be R(0.125) • 1.1185 
as listed in table 4(a). This value of the regret function shows that for the 
worst value of no(nQ “ 0), the relative prediction standard error is Increased 
by at most about 12 percent over what it would have been if the worst value of 
6 had occurred and the best strategy against it had been used. Thus, the 
security regret strategies (for each of the values of ng) must be concluded 
to be widely useful strategies. 


Variance of Predicted Squared Error 

The strategy selections described in the preceding section are based on a 
Monte Carlo investigation that reported mean values of prediction errors over 
1000 simulations. The quoted results thus tell what the mean long run results 
will be as a function of strategy selection. The subject of short run results 
was not discussed. Some insight into the short run performance can be gained 
by examining the observed values of This quantity gives the observed 

variance, for samples of size 1000, of the maximum squared prediction errors 
over the simulations, as defined by equation (28). If this variance is rela- 
tively small, then operating characteristics such as Cgg nix^®^ relatively 
constant from simulation to simulation. But, if V(e2)^j^ is relatively large, 
then the short run performance of a strategy could be erratic. 

In the case of large coefficients of variation (small values of 0) the 
strategy performance was not erratic - the values of were small for 

all of the strategies of table 4 for 0 = 0.125. The strategy performance can 
be erratic for small coefficients of variation (large values of 0). Thus, 
the values of V(e2)n,x were large or small for 6 * 2.000, depending on the 
strategy (table 4). This response to 0 shows that the bias component is the 
component of the prediction error that can be erratic. In particular, the 
values of V(e2)^ were large for 0 = 2.0 when strategies were used 
(table 4) that would result in the smaller values of C__ (0.125). Thus, 

a strategy favorable to large coefficients of varia ion should never be used 
if the possibility exists that the coefficient of variation might be small. 

In such a state of prior knowledge, the security regret strategy for the given 
ng should be used because the V(e2)jjjj^ for it (table 4) was never very large. 


Expected Number of Terms Retained 

Some insight into the operation of the proposed strategies can be gained 
from an examination of the mean number, of terms retained as a function of 
Oq, 0, and the choice of strategy. The results for the admissible strategies 
were given in table 4 and are summarized in table 5. Briefly, the strategies 
that minimize Cgg mx^®^ 0 = 2.0 simply retain many terms for both 

0 “ 2.0 and 0 » 6.125 unless ng is relatively large (uq >3). In a some- 
what similar manner, the strategies that minimize C__ for 9 “ 0.125 

f mx 

are also insensitive to 0 (the value of p remains small for both 9 ■ 0.125 
and 0 * 2.0 unless ng is relatively large (ng > 3). 
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By way of contrast, the security regret strategy results In W being 
responsive to 6 merely provided no > 0. Thus, the results In table 5 
tend to confirm the previously described results, namely, that nn ■ 1 is 
an efficient value of oq, and that the security regret strategy is a widely 
useful strategy. 


CONCLUSIONS 

An Investigation was conducted to determine what statistical techniques 
should be used for model fitting to the results of a two-level, fixed-effects, 
full or fractional-factorial, orthogonal experiment with 16 hypercube treat- 
ments and zero to six center points when the population model coefficients have 
an unfavorable distribution of relative values. Sequential deletion strategies 
using both the F- and a Uj-dlstrlbution and combinations of them were evalu- 
ated, using Monte Carlo techniques, under the criterion of minimizing the 
maximum prediction error, wherever it occurred, among th^ hypercube points. 

Three classes of strategies were identified as being appropriate, depend- 
ing on the extent of the experimenter's prior knowledge. In almost every case, 
the choice of the strategy was found to be unique, according to the number of 
center points. Among the three classes of strategies, a security regret class 
of strategy was demonstrated as being widely useful, in that over a range of 
coefficients of variation from 4 to 65 percent, the maximum prediction error 
was never increased by more than 12 percent over what it would have been if 
the best strategy had been used for the particular coefficient of variation. 

Relative efficiency, when using the security regret strategy, was examined 
as a function of the number of center points, over the range from zero to six, 
and was found to be best when the design of the experiment added only one center 
point to the 16 factorial points. 
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APPENDIX A 
SYMBOLS 


Mathematical 

symbol 

FORTRAN 

name 

Description 

H 

B(l) 

estimate of 

^ae,mx 

ADCOER 

adjusted coefficient of error, eq. (31) 

^ee,mx 

COERMX 

ratio of maximum prediction error to scale 
parameter, eq. (30) 

E(. . .) 


expectation of . . . 

e 

RN(I) 

single observation random error 

i2 

max 

ERSQMX 

maximum over hypercube of mean square pre- 
diction error over simulations 

g 


number of Independent variables 

g-h 

LG^B^ 

experiment contains 2®”^ treatments 

h 


experiment contains (1/2)^ times number of 
treatments in full factorial experiment 

i.j.k 

I.J.K 

subscripts 

£,m,n 

L,M,N 



KODE 

amount of NAMELIST output desired 


KPF 

index number for ap 


KPU 

index number for ay 


LTH 

number of 0 values investigated in any 
computer run 

mp 

MI' 

number of mean squares pooled before test- 
ing begins 

"0 

NO 

number of center points 

"c 

NC 

number of hypercube points 

"t 

NT 

total number of observations in one 
experiment 

"e 

NE 

number of simulated experiments in any 
strategy evaluation 


RK 

RETA 


VESQMX 


YOBS (I) 
Z(I) 


B(I) 

DELTA (K) 


ETA 

THETA (L) 


YMH(I.L) 


dlHCrlbution transfer parameter, eq. (15) 

number of terms deleted Is integer value 
of r,^ times number insignificant 

variance of . . . 

maximum over hypercube of sample variance 
of mean square prediction error over 
sitmtlations, eq. (28) 

independent variable 

conceptual value of dependent variable 

estimate of response function from fitted 
model 

observed value of dependent variable 

mean squares in Yates' order 

nominal significance level of F test 

nominal significance level of Uj test 

regression coefficients in Yates' order 

parameter determining relative magnitudes 
of coefficients in population model, 
eq. (17) 

expectatim of order statistic of 

a variable 

number of mean squares having noncentrality 
parameter of zero 

number of mean squares concluded to be null 
during any analysis 

scale parameter 

mean over experiment of non';entrality param- 
eters, eq. (17) 

noncentrality parameter 

population mean value of Yj for 1^^ 
treatment 


RHO 


number of coefficients concluded to be non- 
null in any sitmilation 
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AVRHO 


mean number of coefficients concluded to 
be non-null in a strategy investigation 

standard deviation of e 
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APPQiDIX B 

DERIVATlffli OF EQUATIONS FOR SUMS OF SQUARES 

Cate a (no ■ Op mo ■ 0). - This case provldea no inforaation for a siaa of 
aquares for a teat ataWatic. 


Caae b (no • 1, Bp ■ 0). - Let Yok be the value obaerved at the k-th 
center point (origin) obaervation. Then by the definition of 

V(Vok> “ ^ < 81 ) 

Alao, fron the definition of o^, where Y^ is the i-th hypercube observationp 

V(Yi) - (B2) 

The object ia to eatimate fiom the information in the yj.» i ■ I, n^, 

and a single center point observation, yQj^. Because the model coefficient 
estimates in the two-level fractional factorial experiment are orthogonal, the 
least squares esthsates of the regressiont coefficients from the combined data 
are all the s«oe as the Yates estimates, except for the coefficient of the zero 
degree term, b^* Its least squares estimate is fron equation (5): 



For any of the treatment points, let Aj be the difference between the 
observed value and the predicted value of Y where Aq is the center point 
difference and i ■ 0, 1, •••, n^. The Yates' estimate of is 


i-i 

The predicted values under least squares estimation are therefore all au^aented 
by b* - bj over their Yates' method predictions. 

The differences between the Yat»' method predictions and the observations 
are all zero at the hypercube p<^ K.ts, therefore over the ng + I treatment 
points, 

* 

^0 ■ FOl • 

- b| - b* i • 1, .... nj. 

The estimate of o^ from the residual of the least squares regression is 
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where 



i»c «»c 

yol £ yi ‘^^01 “ £ yi 

. * i-i 1-1 

yoi - h “ yoi — 


and 


bi - bj 


nc 

L >'* 

i-1 


nc 

yol ^ ^ yt 

i-1 


nc 

s 


yi - "cyoi 


l + nj. "c^^ "c^ 


Thus, 


2 nc 


- (yol “ *»1> "c^^l ■ *’1^ ■ f "n " ^yoi “ ^0 

For ng • 16. 

s2 - 0.941176(yQj - b^)^ 

For thia case, the number of degrees of freedom is 

«dfb • 1 


and the sum of squares is 


SSb ■ Sjndfb - 0.941176(yoi - bj)^ 


(B3) 


(B4) 


In this case, the error sum of squares was obtained from a residual involving 
b|. This usage of b* has the disadvantage that it will introduce a bias or 
"lack of fit" component into the sura of squares if the fitted model is biassed 
at the center point. Becmise of this bias risk, a "pure error" sum of squares 
will be canputed if ng > 1. 

Case e (ng > 1, mp » 0) . - This case is treated as follows. Let the center 
point observations be ygj^; k • 1 ng. Their sample mean is 


^0 


*'0 



yok 


t 
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and the sum of squares, SSg for case c is now; 


SS, 




(B5) 


where the number of degrees of freedom, n^^^ is 


”dfc “ "0 ■ ^ 


(B6) 


Case d (ng =0, mp ^ 0) . ~ bet 


m. 


SS 


“■p 

^ ■ k 


(B7) 


(B8) 


The number of degrees of freedom, n^f^ is 

‘'dfd “ “p 

Case e (ng ■ 1. mp > 0). - This is the additive situation of cases b and d: 


SSg - SS^ + SS^j 


"dfe * "dfb "dfd “ ^ “p 


(B9) 

(BIO) 


Case f (no > 1, mp > 0). - This case is additive with respect to cases c 


and d: 


SSf - SSj, + SSj 


"dff * "dfc "dfd = no - 1 + ®p 


(BID 


(B12) 
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APPENDIX C 

DESCRIPTION OF COMPUTER PROGRAM 

Computations were performed using the FORTRAN~4 program, POOL9U listed in 
appendix D. The antecedents of the program were P00L3U (Holms (1966)), POOLMS 
(^ling and Holms (1973)), POOLES (Holms (1974)) and POOL6U (Holms (1977b)). 
The program POOL9U is outl ><d and the parts that are essentially the same as 
the earlier programs are i» m ified by the section numbers and titles of 
appendix D in the table that follows. The table is followed by a description 
of POOL9U. Illustrative output is given in appendix E. 


Section 

Section 

Reference 

number 

title 

program 

lA 

DECLARATIONS AND TABLES 

POOLMS 

IB 

INPUTS AND CONSTANTS 

P00L6U 

1C 

POPULATION MEANS 

P00L6U 

ID 

STRATEGY 

(new) 

2 

SllflJLATIONS AND MODEL FITTING 

POOLES 

3 

CONSTRUCTION AND ORDERING OF MEAN SQUARES 

POOLMS 

4 

DELETION OF TERMS 

(new) 

5 

PREDICTIONS 

POOLES 

6 

ACCUMULATION OF ERRORS 

POOLES 

7 

DETERMINATION OF MAXIMUM AND MEAN SQUARED ERRORS 

POOLES 

8 

OUTPUT 

(new) 

9 

YATES METHOD SUBROUTINE 

POOLMS 


Section lA. - Declarations and tables . - The values of the nominal test 
size a are stored as (ALPHA(I), I = 1,11) and later used as output labels. 
These values range from 0.001 to 1.0, however, the value of 1,0 obtained by 
setting the index to 11 is merely a code implying that no significance testing 
is performed. 

The sequential deletion requires critical values gainst which the test 
statistics are compared. The critical values of F are stored internally as 
((FTB(I,J), J “ 1,10), I = 1,20) where I Indexes on the degrees of freedom 
and J Indexes on the value assigned to a. The critical values of Uj are 
stored internally as ((TB(I,J), J = 1,10), I = 1,16) where I is the order 
number in nondecreasing order, and J Indexes on the value assigned to a. 
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Section IB. - Inputs and constants . - The constants defining the popula* 
tions, the experiments, and the sequential deletion strategy are read from 
data cards in the following order, with the order of the fields being the 
same as the order of the symbols In the following description. 


Format 

Description 

(13A6,A2) 

REMARK (I), arbitrary literal Information such as particular 
use of program, date of last change, and so forth. 

(315) 

LGMH, NE, KODE 

(I8,5F8.3) 

LTH, (THETA(L), L • 1, LTH) 

(I4/(10F8.5) 

NDELTA, (DELTA(K), K - 1, NDELTA) . There are as many (10F8. 
cards as are necessary to read (DELTA(K), K • 1, NDELTA. 

(812) 

NNO, (N0(M), M * 1, NNO) 

(3I5,2F5.3) 

MP, KFF, KPU, RF, RETA (The associated READ statement is 
actually in section ID.) 


Section 1C. ~ Population means . - After the initial constants have been 
read, the next major operation Is the formation of the population mean values. 
The number of population regression coefficient sets to be examined during the 
investigation of a strategy is the number, l0, of 0-values. 

With respect to equation (2) all the population model parameters are first 
set equal to zero with the DO-loop ending at statement 10. The non-zero values 
of 8^+1 are initially set equal to using the DO-loop ending at statement 

20. The DO-loop ending at statement 20 serves the purpose of equation (19) with 
0 ® 1 and 0 « 1. The value of o = 1 is retained, but the adjustment for 0 
is made after the population mean values have been computed. 

With the population 6-values (aside from 0) established at statement 20, 
the object is to compute the population mean values from the 6-values by the 
reversed Yates' method (Duckworth (1965)). The first step is to reverse the 
order of the 6-values, which is completed at statement 22. The use of the 
reversed Yates' method then yields the array YOBS (I) as completed at statement 
30. The array YOBS(I) is therefore an array of population means This 

array of population means is to be expanded over the mean noncentrality param- 
eters, Xji, to give the effect of equation (17). This effect is produced on the 
population mean values by the multiplication 

Vi,l - Wi0n 

and this operation is completed with the creation of the array YMU(I,L) at 
statement number 48. The values of uij are thus indexed over treatments 
i, 1 “ 1, ..., n^, and over arbitrary values of 0^; i ■ 1, ....ig. 

The index, 1, runs over the mean squares to be analyzed within a single 
experiment and thus unequal values, contribute to non-uniform noncentrality 
parameters within the experiment. The Index, i serves to change the scale of 
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the noncentrality parameter and, therefore, each successive value of I gen- 
erates a new family of experiments. Changing 6^ thus provides the conditions 
necessary to Investigate the deletion procedures for differing coefficients of 
variation. 

Section ID. - Strategy . - In terms of mathematical symbols previously de- 
fined, the strategy parameters are functions of numbers that are read at state- 
ment 50 as follows: 


Argument 
FORTRAN symbol 


Function 


Mathematical Symbol 


MP 

mp 

RF 


KPF 

Op 

KPU 

“u 

RETA 



More than one model deletion strategy can be evaluated during any computer 
run. On completion of the evaluation of a particular strategy, control is 
transferred back to statement 50 for the reading of an additional strategy data 
card. The operation of the program ends when such cards are exhausted. 

The error simulations are generated so that all strategies are compared 
for the same set of random numbers. This is achieved by reinitializing the 
random number generator for each new strategy with the statement "CALL SAND(XS)." 

The prediction errors and their squares are stored in the arrays ERSQ 
(I,L,M) and ERSQSQ(I,L,M) . These arrays are initially cleared by the loops 
terminating at statements 97, 98, and 99. 

Section 2. - Simulations and model fitting . - The number of experiments 
simulated is NE. The performance of these experiments and their analysis is 
controlled by the loop; "DO 699 N = 1, NE." Within each experiment, the 
random numbers for the (n^ + Oq) "observations" are generated as follows. 

The procedure generates a sequence of pseudo random numbers with a rectan- 
gular distribution by taking the low order single precision bits of the product 
rr_i*K where rj_ 2 ^ = previous random number and rg = 1 and K *= 5^5, This 
fixed point number is then floated and returned to the calling program as a 
floating point number between 0 and 1 (Tausky and Todd (1956)). 

The rectangular variates are transformed to pseudo-normal variates using 
a procedure described by Box and Jtiller (1958). The procedure begins with 
and D 2 assumed independent and rectangular on the interval (0,1). In the 
notation of Box and Muller (1958), the transformations are: 
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1/2 

Xj^ - (-2 In Dj^) c08(2ttD2) 
1/2 

X2 * (-2 ?,n sin(2irD2) 


The operations are completed at statement number 215. 

Each set of random numbers for an experiment is used with all values of 
the population and design parameters 0 and ng through the statements 
"DO 690 L = 1, LTH." For all of these cases, the simulated observation errors, 
as stored in RN(I), are added to the population mean values (stored in YMU(I,L)) 
for the particular treatments (1 = 1, ...,NC), at statement 22A as required by 
equation (22). (Beyond n^ an additional ng values of RN(I) are used as 
"center point" observations.) 

After synthesizing the "observed" values of YOBS (I) the "SUBROUTINE YATES" 

(section 9) ending with statement 909 is used to compute the array (B(I) which 

contains (except for division by the number of treatments) the Yates estimates 
of the parameters in the manner of equation (3) and in the order of equation (2). 

Section 3. - Construction and ordering of mean squares . - The mean squares 
are formed from the parameter estimates (for those terms beyond Bj^) and a 
pointer function is created within the loop "DO 309, 1=1, NC." As exhibited 
by figure 1, the array BFM(I) remains intact for m = 1, .•■, 0^0, but changes 

as I = 1, ...,S.g. (In section 4 the array B(I) will be overwritten for all 

m = 1, ... <nj^Q) • 

The array of pointers to the B(I) array is created by the statement 
IND(I) = I. This array will serve to identify the coefficients in the B(I) 
array after the process of ordering mean squares according to rank. The order- 
ing is done in the sequence of statements ending with 313. 

Operations thus far created a column of mean squares Z(J) with mean squares 
Indexed on J in the order of increasing rank, together with a column of integers 
IND(J) Indexed on J. Thus, any address J will lead to a mean square Z(J) and 
also to the integer IND(J). This integer is the index 1 that the associated 
regression coefficient has in the original Yates' order. 

The computation of the sums of squares is done for each value of oq within 
the loop: "DO 680 M = 1, NNO." The construction begins following statement 313 

and ends with statement 365. The operations are outlined by figure 2(a). 

The computation of the sums of squares depends on the values of Oq and 
mp according to cases b, c, d, e, and f of table 1. Three combinations of 
tnese cases are identified in the statement immediately preceding statement 320. 
If nQ = 0, the situation is that of case a or d, and control is transferred to 
statement 330, following which the SS^ of equation (B7) is evaluated at 
statement 365. 

If nQ = 1, the situation can be that of case b (oq = 1, m ■ 0) or case 
e (oq = 1, mp > 0) and the SSj, of equation (B4) is computed at the statement 
following 325. 
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If hq > 1, the situation can be that of case c or case f. The quan- 
tity SSp of equation (B5) is computed at the statement for TO! that follows 
statement 322. 

If mp and no are each zero, there can be no sequential deletion, and 
control is transferred to statement 432, and all terms are retained. Setting 
both Op and oy equal to 1.00 (by setting KPF « KPU “ 11) is used as a 
code signifying that no conditional pooling is to be done but that arbitrary 
deletion is to be accomplished according to values assigned to iRp and r . 
This is done by transferring control to statement 421. 

Section 4. - Deletion of terms . - The flow chart for the tests of signif- 
icance is shown by figure 2(b). The procedure begins at statement 417 and ends 
at statement 419 (appendix D). The significance tests will have been avoided 
by earlier statements in section 3 if either ny + mp = 0 or both ap “ 1.0 
and Oy * 1.0. Thus, entry at statement 417 requires both ny mp > 0 and 
at least one of «p or Oy < 1.0. If Up = 1.0, control is transferred to 
the U| test which begins at statement 418. If ap < 1.0, control is determ- 
ined (fig. 2(b)) by the questions: "Is j > rpny? and is j > 1?" If both 
are "yes," control is transferred to statement 418, which Initiates the Uj- 
testing. 

Irrespective of whether significance testing is against the F-distribution 
or the Uj-dlstribution, insignificance pools Z(j) into the denominator of the 
test statistic and then transfers control to statement 419 which increases j 
by one unit. Significance at any j transfers control to statement 420. 

The third statement following 417, namely the statement "IF (KPF.GT.IO) 

GO to 418" transfers control to the Uj-test merely provided op = 1.0, even if 
j < 2. But, the rationale of the Uj distribution leaves Uj undefined for 
j < 2. The possibility of a transfer of control to the Uj- distribution with 
j * 1 was provided for by setting the critical values of Uj equal to 2.0 

for j = 1 and all values of ay < 1.0. Thus, if j = 1 then obviously 

fflp * 0 and the test statistic is (from table 1) 

2-0 2.0 
“1 “ SSy + = 1 + SSb/z(i) 

for case b(ny = 1, nip = 0) or 

1 SS^ + 

for case c (ny > 1, mp = 0). Thus, for case b, u^^ ± 2.0 for SSb i 0.0 

and ZQ) would not test as significant. For case c (ny ^ 1, mp = 0), 

uq) > 2.0 only if 

SSc + -(1) 
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hence only If 


2.0 SSc 

^(1) ^ (nc; - T^ 


Let be estimated by SS(./(no - !)• Then u(i) > 2.0 only if 


*(1) 


^ 2.0(no - 1) 

(no - 2.0) 


SS„ 

(no - 1) ' 



The following table shows C(nQ) as a 
appropriate to case c. 

"0 

2 

3 

4 

5 

6 


function of Oq for the values of nQ 


C(no> 


4.0 

3.0 
2.67 
2.5 


Thus, z(i) (which is the smallest of the ordered mean squares) would have to be 
much larger than before ^{\) would be declared significant. 


The flow chart for the model deletion and for the estimate, p, is shown 
by figure 2(c). Transfer of control to statement 420, 421, or 422 leads to 
the estimate, respectively; 


A 

n = integer i r^(j - 1) 


or 


n = integer <. r^^mp 


or 


n = integer <. r^(n(, - 1) 

With n so estimated, the n smallest absolute value coefficients 
(beyond bj^) are set equal to zero with the statements ending at 425. 

Section 5. - Predictions . - Predicted values of the dependent variable 
for all the treatments of the fractional factorial experiment are computed in 
this section using SUBROUTINE YATES' and the reversed Yates' method as proposed 
by Duckworth (1965). 

The operation of Yates' method followed by the "reversed Yates' method" 
is illustrated by the following table for a 22 experiment ; 
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YOBS 

Y, 


Yi + Y2 

^ 2-^1 
Y 4 - Y 3 


YATES METHOD 
B 


Yi + Y 2 + Y 3 + Y^ 
Y2 - Yi + Y^ - Y3 
Y3 + Y^ - Yi - Y2 
Y 4 - Y 3 - Y 2 + Yi 


B/FNC 

(Yj + Y 2 + Y 3 + Y^)/4 
(Y 2 - Y^ + Y 4 - Y3>/4 
(Y 3 + Y 4 - Yi - Y 2>/4 
(Y4 - Y3 - Y2 + Yi )/4 


REVERSED YATES METHOD 


YOBS 


YPRED 


(Y4 “ Y3 - Y2 + Y^)/4 

(Y4 - Y2)/2 

Y4 

^1 

(Y3 + Y^ - Y^ - Y2)/4 

(Y2 + Y4)/2 

^3 

^2 

vY2 - + Y^ - Y 3)/4 

(Y3 - Y^)/2 

^2 

^3 

(Yi + Y2 + Y3 + Y4)/4 

(Yi + Y3)/2 

^1 



In the case of the computer program, there are parameters estimated from a 

fractional factorial experiment. 


Section 6 . - Accumulation of errors . - The squared error for each pr«ilc- 
tlon Is accumulated (as required by eq. (24)) In the array ERSQ(I,L,M) as 
computed with the loop "DO 609 I “ 1, NC." These accumulations are stored for 
each combination of L and M as Indicated by the loops terminating at state- 
ments 680 and 690, and this process Is repeated for each of the n^ sets of 
random numbers as Indicated by the loop terminating at statement 699. For the 
purpose of computing the variance of the squared error of prediction, the 
quantity 

^ ^®pilmn^ 


of equation (27) Is computed within the loop ending at statement 609 and stored 
as ERSQSQ(I,L,M). 

Section 7. - Determination of maximum and mean squared errors and their 
variances . - The purpose of this section is to determine maximums and means of 
the prediction errors over the space of the experiment after the errors have 
been evaluated over that space by accumulating over the simulations. The 
accumulation over the number, n^, of simulations had been stored in the array 
ERSQ(I,L,M). For particular L, and M, the determination of the largest pre- 
diction error over the space of the experiment as defined by equation (25) Is 
done through repeated use of the library subroutine AMAXl, which determines a 
real number as a function of two real arguments. This is done within the loop 
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"DO 750 I " 1, NC." The summation for the mean squared prediction error over 
the space of the experiment as required by equation (26) is also done within 
the same loop terminating at statement 750. After division by the appropriate 
divisors, these two evaluations of error are stored in the arrays (ERSQMX(L.M) 
and AVERSQ(L,M). The quantity 


«e 

^®pUmn^ 

n»l 


2 




2 


is computed and stored as TEM within the loop ending at statement number 750. 
The quantity V(e|^) defined by equation (28) is determined to be the max- 
imum of the values o?^?EM as determined by 

E » AMAXl(E.TEM) 

and from this maximum, V(e?„) is computed and stored with the statement 

im'max 

VESQMX(L,M) » E/FNEMl 
The sura of the values of TEM as given by 

F = F + TEM 

is then used to compute V(-e^j^) according to equation (29) using the statement 


AWESQ(L,M) « F/FEMINC 

The computation ends if the data for MP, KPF, KPU, RF, and RETA, are 
exhausted; otherwise a new strategy is investigated by returning control to 
statement 50. 

Section 8. - Output . - The output is Illustrated in appendix E. The 
NAMELIST output was incorporated only for program checking. 

Section 9. - Yates' method subroutine . - This subroutine is essentially 
that of part of the main program of POOLMS (Amling and Holms (1973)) except 
with the last few statements modified so that the subroutine can be used for 
the direct Yates' method and also for the reversed Yates' method; as was also 
done in POOLES (Holms (1974)). 

The algorithm for Yates' method is described as follows: The "observa- 

tions" yj j may be visualized as a column (j “ 1) with row index i ■ 1,...,2*'. 
The column"^ is then operated on according to Yates’ method to produce a succes- 
sion of columns j 2,...,£. The successive columns for any row are 
computed as follows: 

Ti = 1. 3, 5,.. .,2^ - 1 

^k.j ' ^i+i.j-1 ^l,j-l S 

1 k = (i + l)/2 
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y 


k.j 




i 

k 


1 , 3 , 5 ,.. .. 2 ^ - 1 
(2^ + t + l)/2 


New columns are computed according to the two preceding equations for 
j ■ 2,..., K.(to create <■ columns). 


o o o 
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APPENDIX D 

LISTING OF COMPUTER PROGRAM POOL9U 


C 

c 

c 


II.- SiCUlRlTlONS ANU TABLES 

DI<1ENSI3N rtiHA^KCm). AL»HAIll)f TsU6«lb)i RNI24I, XN3I ISI t 2US 1 1 
ITMETAIS). KC*(7), OlLTMIS). YHJU6,SIi AVRHOISfT). ERSOtlSfStT)* 
2LRSCS0(16,S«7l,iRSLMX(St7l. C3E^MX(5,7it AVERSaiS.?)* 

3VESQNX(5.7)f AWESQIS.7). A0C0£R(5,7I, ?FN 1 16 ) « ^T3 ( 70. 1 0l 

C3^N09 KA. Y09S(16I» 0(161 

DATA (IL^HAdl .Ul.lD/S.Sm.a.iaZ.n.OSS.O.Ol.w.OZSt o.as.o.tu, 
ic.26.a.50f Q.7E, l.C/ 

0 ITA((^T 3 ( If J).J = l.iC) fi:it 23 )/ 4052 ly. 3 ( 10 l 3 Z 1 . 3 .l 62 n.?.H 0 S 2 . 2.66 
A 7 .B. 161 . 4 . 39 . 06 i 5 . 62 b. 1 . 333 . 3 . 1716 . 995 . 5 . 490 . 5 . 198 . 5 . 96 . 53 . 75 . 51.1 
06 . 51 . 8 . 526 . 2 . 571 . C. 666 7 . 3 . 1333 . 167 . 3 . 134 . 3 . 55 . 55 . 34 . 12 . 17 . 44 . 13.13 

Cf 5.5 36 . 2 . 0 ^ 4 .n.<;a 5 l.iJ.l 223 f 74 . 14 . 51 . 45 . 3 l. 37 . 21 . 20 . 12 . 22 . 7.7 09 . 4 .S 

045 . 1 . 837 . 0 . 5456 . 0 . 1165 . 47 . 16 . 34 . 73 . 22 . 78 . 16 . 26 . 10 . 01 . 6 . 605 . 4 . 060.1 

E. 692 . L. 528 1 . 0 . 1134 , 35 . 51 . 27 . 12 . 16 . 64 . 13 . 74 , 8 . 513 . 5 . 967 . 7 . 776 . 1.621 

F. 0 . 5 149 , 0 . ill 3 , 29 . 24 , 22 . 93 , 16 . 24 . 12 . 25 , 3 . 07 3 . 5 . 591 . 3. 56 9 . 1 . 8 73 . 0.5 
60 S 7 , 0 . 1 09 9 , 25 . 4 ?. 20 . 26 , 14 . 69,1 1 . 25 , 7 . 571 . 5 . 318 . 3 . 459 , 1 . 533 . 0 . 4990 . 
HO. 1068 . 22 . 66 . 16 . 46 . 13 . 6 l. IQ. 56 . 7 . 2 m 9 . 5 . 117 , 3 . 3 oO. 1 . 512 , 3 . 4938 , 3. 13 
160 , 21 . 34 , 1 7 . 17 , 12 . 63 , IQ. 04 . 6 . 937 . 4 . 9 » 5 . 3 . 265 , 1 . 492 , 3 . 4897 . 0 . 1073.1 

39 . 69 . 16 . 20 . 12 . 23 . 9 . 646 . 6 . 724 . 4 . 844 . 7 . 225 . 1 . 475 . 3 . 4654 . 0 . 1368 . 18.64 
R, 15 . 44 . 11 . 75 . 9 . 3 30 . 6 .F 34 . 4 . 747 , 3 . 175 , 1 . 461 . 3 . 4837 . 0 . 1063 , 17 . 62 , 14 . 
L 64 .il. 37 . 9 . 074 . 6 . 414 , 4 . 667 . 3 . 136 . 1 . 480 . 3 . 4614 . 3 . 1059 , 17 . 14 . 14 . 34.1 
H 1 . 36 , 3 .b 52 . 6 . 298 , 4 . 6 w 0 . 3 . 102 . 1 . 4433 .Q .4794 .n.lQS 6 . 16 . 59 . 13 . 94 . 13.8 
N 0 . 9 . 663 . 6 .L 00 . 4 . 543 . 3 . 373 . 1 . 432 . U. 47 7 3 . 3 . 1053 , 16 . 12 . 13 . 89 , 13 . 58 , 8 . 
0531 , 6 . 115 , 4 . 494 , 3 . 048 . 1 . 425 . 3 . 4763 . 3 . 1351 , 15 . 72 . 13 . 29 . 13 . 36 . 6 . 43 y, 
Pb. 34 :, 4 . 4 5 13 . 3 . U 26 . 1.4 19 . 3.4 750 , 3.13 49 . 15 . 38 . 13 . 03 , 10 . 2 ?. 3 . 785 . 5 . 9 
G 78 . 4 . 414 , 3 . 007 , 1 . 4 13 u, 3 . 4 7 36 . 3. 104 7 , 1 5 . 3 b. 12 . 81 . ID. 37 . 3 . IS 5 . 5 . 922 . 

94 . 36 1 . 2 . 99 0 . 1 . 406 . 0 . 4728 . 3 . 134 5 . 14 . 32 . 12 . 62 . 9 . 944 . 8 . 096 . 5 . 872 . 4. 35 

51. 2 . 97 5 . 1 . 404 . 0 . 4719 . 0 . 1044 / 

DATA ((T 8 (lfJ).J=l, 10 ).I= 1 . 16 )/ 2 . 0 ,?. 0 .?. 0 .?. 3 , 2 . 0 , 2 .Q.?. 3 , 2 . 0 . 2.0 
1 . 2 . C, 2 . 0000 . 1.99 999 . 1.99 997 , 1 . 99966 . 1 . 9991 7 . 1 . 9 9687 . 1 . 9877 , 1 . 923,1 
2 . 706 , 1 . 38 2 , 2 . 9 976 ,^. 9960 , 2 . 9904 , 2 . 93 09 , 2 . 961 , 2 . 904 , 2 , 836 , 2 . 827 , 7.3 
366 , 1 . 68 b, 3 . 976 , 3 . 962 , 3 . 925 , 3 , 970 , 3 . 760 . 3 . 625 . 3 . 412 , 2 . 949 , 2 . 395 , 1.9 
4 olf 4 . 6 S 7 . 4 . 848 . 4 . 738 . 4 . 65 . 4 . 44 ,b. 2 i. 3 . 3 ‘^. 3 . 267 . 2 . 6 bS. 2 . 1 B 4 .^. 74 . 5 . 
56 3 , 5 . 46 , 5 . 31 , 4 , '} 9 , 4 . 68 . 4 . 28 . 3 . 67 , 2 . 393 . 2 . 371 , 6 . 51 , 6 . 33 , 6 . 11 . 5 . 87. 5 
6 . 46 . 5 . 09. 4 . 61 . 3 . 33 . 3 . 11 , 2 . 54 . 7 . 20 . 6 . 96 . 6 . 66 . 6 . 35 . 5 . 96 . 6 . 44 . 4 . 91 . 4 . 
706 , 3 . 29 , 2 . 69 , 7 . 81 , 7 . 67 . 7 . 1 C, 6 . 78 . 6 . 26 , 8 . 75 . 5 . 17 . 4 . 27 , 3 . 45 . 2 . 62 . 8. 3 
B 4 . 9 . LI. 7 . 5 3 . 7 . 17 . 6 . 59 , 6 . 03 . 5 . 41 , 4 . 45 . 3 . 6 U. 2 . 95 . 3 . 82 . 8 . 44 . 7 . 95 . 7 . S 3 
9 . 6 . 69 , 6 . 26 , 5 . 61 . 4 . 62 . 3 . 74 , 3 . 07 ,°. 26 . 8 . 34 , 8 . 33 . 7 . 87 . 7 . 1 ^, 6 . 5 ^, 5 . 81 . 
A 4 . 77 , 3 . 67 , 3 . 17 . 9 . 67 . 9 . 21 . 3 . 66 . 0 . 1 b. 7 . 37 . 6 . 71 . 5 . 99 . 4 . 92 . 3 . 99 . 3 . 27.1 
90 , 35 , 9 . 55 , 6 . 98 , 8 , 42 , 7 . 59 , 6 . 91 , 6 . 15 , 5 . 05 . 4 . 10 , 3 . 37 , 10 . 43 . 9 . 86 , 9 . 20 , 
C 6 . 66 . 7 . 79 . 7 . 07 . 6 . 30 . 5 . 17 . 4 . 20 . 3 . 46 . 10 . 7 ?. 10 . 14 . 9 . 43 . 6 . 63 . 7 . 96 . 7. 23 
D. 6 . 44 , 5 . 29 , 4 . 30 , 3 . 55 / 


SAhELIST /UUTl/ o /0UT2/ Y09S /DJTi/ Y4J /0UT4/ INDXK /''UT5/ RS 
NAH'LIST /uUT6/ INDX4 /DUT7/4T /0JT3/ INQXL /0jT 9/ IN0,2 
NA4-LI5T /yUTlw/ NDF,Tl 4 /OUTli/ /0JT12/ SSjYO /OuTi4/ £TA 
NA45LIST /O’JTU/ l4wXJ, T£M, NjF, TEST, JN /QUTIS/O-TA 
namelist /0JT1&/ K40 /DUT17/ f953 /DJT18/ L9S3S3 /0JT19/ ^T^NT 
NiMELiST /0UTZ3/ KODE /DUT21 / 8FM 
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i9«* Xi«iPUT6 AN3 CO^SIASTS 

RCAOIStSaU) IP£**ARAm.lriatl 
ilRIT£(6,8Uit (ReHARK<II,i:i»lA) 

READ (5,SO^i 13HH, f«it K90C 
IP IK03E .&T« li ) mRITE I6i 0UT2Q} 
REASiStSOR} Lt-i% (THiTAtU, L^IiLTh) 

READ (5t3G6i NDEUTA, (gCLTAINl, K = l»MDEt.TAl 
hRITE U»8U7) SDELTA, (OEuTACKIt k:}*NDELTA} 
READ ($«81U) NRilt (Nu<M)t m:1,NN3I 
HK : L3M^ 

HZ = 2**L3HH 

WRITE <6t6c3l <K» NC| SS3i ME 

NCHl : M:-i 

NCH2 S m:-2 

NCPi = m:*a 

NTM* : m: ♦ MCCMNQI 

MTHXPl : MTHX ♦ I 

NEMl s NE • I 

fn: : m: 

FNE r me 
FMEMl : MEMl 
FME3NC : Mt*NC 
FEMINC = NEMl*MC 

ic,- pdpulatiom meams 

00 1C 1=1, n: 
sill : O.C 
10 COMTINJE 

00 20 I=1,NDEITA 
d(I*l) : OcLTAtn 
20 COMTIMUE 

IF U03E .OT. u ) WRITE lb* DUTl ) 

00 22 I^lfNC 
n:»1MI = MC»1-I 

voasm : BfMCPtMij 

22 COMTINJE 
call TATES 

IF (K33E .6T. a I WRITE (6» OJTl ) 

00 3w i:ifNC 
n:» 1MI r NCPl-I 

Y03sm r btNCPl**!! 

30 COMTlMJE 

3^ IKODE .LT. i I WRITE Ifc, 0JT2 1 
00 48 L=i,LTH 
DO 4 7 1 = 1, k: 

vMun.Li = tobsiiuthetaclj 

47 COMTINJE 
43 COMTlMJE 

IF (K03E .6T, i I WRITE 16, OUT* I 
ID. - STRA TEST 

53 READ l5,S0b,EMD = 8991 , wOF j , Rr , R£T A 

HPPl r M?*i 

call sand iXS) 


f •» r-» ri rt r» rt 
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DO 99 tUlvKNO 
00 99 

AV9H0I Lf 9) : O.J 
00 97 I = i,«C 
CRSUXfLfMU o.n 
£9$US0f ItLtM) : 0,0 
97 C09TINJE 
99 COMTINJE 
99 COSTINGS 

2. - SI1ULATI09S AMO MODEL FITTIMS 

00 995 
INDXN : S 

IF (K03E .CT. 3 ) iiRI^E 16, 0JT4 ) 

00 ?i3 I=l,»jTMxPk 
call RAIOiRNfin 
213 C09TI»iU£ 

IF (K03E .LT, •« I kRIU 16, OUT! ) 

DO ns 1 = 1,STMX,2 
E= SCRTI>2.0 *Al 06(RSCI) n 
0= 6.2S318S3*RNII«i) 

RN(1U E*:OSO} 

E*SIK'(0) 

as CCHTINJE 

IF IK03E .6T. >4 ) mRITE 16, DOTS I 
00 690 L=1,LTM 
INDXL = L 

IF IXC3E .GT, 7 I mRITl 16, OUTS 1 
00 22H 1 = 1, NC 

YOSSm = VMUtI,U « R9(II 
2?4 CONTINUE 

IF (K03E *LT. i ) i«RITE (6, 0JT2 I 
call YATlS 

IF IK03E *GT. U I NUITL (6, OUTl i 

3. - CONSTRUCTION AND ORDERING OF MEAN SQUARES 

00 309 1=1 , NC 
INOlDs I 

ZIII : 3II*1»*BII*1I/FnC 
3FNI1I : 3<I1/FNC 
3D9 CONTINUE 

IF fXOOE .GT, 8 } 2RITE < » , 0UT9 } 

IF IX03E .wT. 3 ) mRITE (6, 0UT21 I 

DO 313 J=i,NC“2 
TEST : 2<NCMII 
IN : NCMl 
DO 312 NA = J,NCH2 
1F(TEST-2I KA»I 312,312,311 
^11 TEST : ZINA) 

7N= NA 

iI2 continue 

item: inOI IN) 
tem: :m) 

INO(IN): IhOIJ) 
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?UM: 

UPUl: irt¥ 

7iJ) t Tf»* 

Hi CCNTI^U*- 

If ll-rcf .OT, 6 I fPITt* I*', Ol'TP ) 

nr frf'i 

iNDXf : H 

ir I^'OIT .GT. J wPITt l», 0UT4 I 

ir *CT, If I wPiTt u, oumi 

re I = i,KC 

!-III r 

;>;6 CONTIKur 

NT : KiC ♦ I 

ir iHCDf .ST. 6 J WRITE Ol'T” I 

FN' : N'lVl 
9fU: - PF*FN. 

TV r : 

IF IN. l-I-ll J3»^ , T*'. 

sstvc : .;. 

00 3.2 I=fCPi,NT 

TV = TV ♦ RMII 

SJCYO ; SitVO ♦ TMI)**: 

3^2 CtNTINUt 

IF iwrt': ,GT, ;n WRITE IF, CUT’. I 
TFM r S5GV0 - I I T V** I /FN »| 

NfF : N IVI-I 

bliJ : IF*C*Mi» ♦ TVI/IFNC ♦ FN I 

5C Tc 

;.5 TV : RNINCPll 

TfM r .“S** : I 7 fc* I I Ty - fFMin»** 2 l 

NEF : i 

Din = lFKC*fcliI ♦ TYI/*FNC ♦ FW.I 
GC TC Z~b 
Hi TEH : . , 

NET = . 

Hi IF IWOC*' .GT, *l WkITF 16, OUTfel 

IF I VODE .CT, ? I WRITE I , ruTg ) 

IF IhOtE .CT, ? I wPITE |^, ClT’.l 
IF IN'I«) ,LT, 1 .*Mi. MF ,LT, n CP TO 
IF IfPF .FT, .INP, l<PU .OT. GO TP «21 

IF li'P .LT, il re TC PJV 
nc 3F* J : \,HP 
INDV.J : J 
TFH r TPM ♦ 2IJI 
Iti CCNTINLE 

NCF r NTF ♦ HP 

IF n^OLr .GT. V I WRITE l», OUT’.I 

C 

C K.- fELFTICN OF 

C 

••iT DC R.V J:»‘PF 3 1 

INOVJ ; J 
FJ : J 

IF IWRF .CT. ?CI fC TO H.P 

IF IFJ ,0''. CFM' INPVJ ,CT. n CP TP PIP 

c 

C F - irST 
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FKDF r NDF 

TfST : FNPF * ZIJ) / TfcM 
IF <>(ODE .6T, 12> WRITE 16, OUTUI 
IF HEST ,GT, FTF IKDF.KPF n GO TO 42C 
TEM • TEM 4 2(j| 

NCF r K'PF 4 1 
GC TC <419 
C 

C UJ - TEST 

C 

416 IF IKPU .FT, IC) fC TO 4ZP 
NDF = NDF 4 I 
JK : J 
FFDF = NDF 

TEST = FNrF*2IJ) /tTEM42|jn 
IF <60DF .GT, id write 16, OUTIS) 

IF ITEST ,5T. TP«JN,KPU)I GO TO 4?Ci 
TEM : TEM 4 2|jj 
419 CONTINUE 

JETA = NCMl 
GO TC 4?2 
42U JETA = J-1 

IF IKODE .GT, 12) WRITE <6, OUTTi) 

GO TO 4?2 
411 JETA r MR 
422 ETA = JFTA 

IF IKODE ,GT, 13) WRITE (6, OUT’4) 
JETA = TFIX <RETA*f TA ) 

IF IKODE ,GT, 14) WRITE IN, OUT!t>) 

IF IJETA ,LT, 1) CO TO 434 

DO 4^5 J=!,JETA 

INDXJ : J 

INDX = INriJ)4l 

BlINDX ): I 

4 T", continue 

IF IhOOE ,GT, 0 ) WRITE 16, OUT’ ) 

GO TC 4 54 
4:2 JFTA r 

434 RHO : NCM’ - JETA 

IF IKODE .GT, IE) WRITE 16, OUTlb) 
AVRHOIL.M) r AVRHOIL.M) 4 RHO 
C 

C 5,- PREDICTIONS 

C 

IF IKODE ,GT, 5 ) WRITE I 6, OUTt ) 

IF I KuDE ,GT, 7 I WRITE I 6, OUTS I 
DC E4f. 1=1, NC 
NCPINI = f*CP;-I 
YCBS (I ) = B INCPINI ) 

S46 CONTlNy 

IF CKODE .GT, 1 ) WRITE 16, OUT? ) 
CALL VATlS 

IF (4-CDE ,GT, n ) t^RT'iE If, OUT 1 ) 

C 

C 6,- ACCUMULATION CF ERRORS 

C 


o r> o 
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DO 69 Ii’.NC 
NCPIHI : WCri-I 
TEH : in<6CPlPI> - YKIU« I iL I 
ERSCII,L,PJ r EPSCn,L,Mj 4 TEM 
ERSC5C I I,L ,H ir FKfCSOntL«M) 4 T£P**2 
{.19 CONTINU*' 
tfc(' CONTINUE 
fr9C CONTINUE 


IF 

(NODE 

•GT. 

16) 

WRITE 

<6, 

OUT ?7) 

IF 

(KODE 

.GT. 

17) 

WRITE 

<6. 

0UT18I 

699 CONTINUE 






IF 

(WODF 

.GT. 

16) 

WRITE 

<6. 

OUT! 7) 

IF 

(KOCE 

.GT. 

17) 

WRITE 

«6, 

ouTtei 


7.- EE termination OF “AVIMUM AND MEAN SCU^RFD ERRORS 


C 

C 

C 


00 79il Mil.NNE 
INDXM = M 

IF (►ODE .6T, 5 ) WRITE ( 6, 0UT6 I 
FNT X NC 4 N. (M) 

RTFNT ; SCRTIFNTJ 

IF (KOCE .GT, 16) WRITE (6, 0UTI9) 

00 7f LrJ.LTH 
INDXL L 

IF (WODE .GT, 7 ) WRITE (»^, 0UT« ) 

C - ■ 

D = . . i,. 

E= 

F: 

DO 7? » in.NC 
C = AMAXilC,ERSC(T,L,M) ) 

D = D 4 EFSC(I,L,M) 

TEM= EWSCSC ( i ,L ,M ) - ( I ERSC ( I ,l ,M ) /FNF 

E : AMAXKE.TEM) 

F : f 4 tEM 
7S0 CONTINUE 

ERSCMX(E,M) : C/FNE 

COERMX(L,M) r ( SC R T ( ER SCM X ( L , M ) ) ) /THE T A I L ) 
AVERSOIL.M) = D/FFEPNC 
VESCMXIL,M)z F/F^EMl 
AVVFSCIL.M) r F/(EMlNC 
AVRHOL.M) : AVPNCd ,M)/FN'E 
ArcOlRIL.M) r COEPHX (L ,M)*RTFNT 
/ct CCNTlNUr 
79i CONTINUE 


•r, - 


U 


'■t'f 


t* * 




■S,- OUTPUT 


WRIT{ (F, fjE) 
WRIT! l6,Eil) 
WRITE (6,Fi3) 
WRITE (6, PIS) 
WRITE (6,f!7) 
WRITE (6,F31) 
WRITE (6,P17) 
WRIT) (6,E3S) 
WRITE ('.,?17) 


(MR, RF, ALPHA(KPE), ALPHA(KPU), RtTA) 

(N ,(M) , Mri.NN j) 

( T)iLTA (L ) t < AVPHO ( L t M ) , M z I , NNC )»L = ltLTH) 
(TH'-TA (L),(FRS(.MX(L,M),MrT,NND ) ,L=1,LTH) 
(THrTA(L),(AVERSC(L,M),M=l,NNr ),Lzl,LTH) 
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C 


WRITE 
WRITE 
WRITE 
WRITE 
WRITE 
WRITE 
WRITE 
WRITE 
&0 TO 
B99 STOP 


t6»817) (THETA(Li,(VCS3HXfLtHI»M=l|NNQ )tL=lflTHI 
18,837} 

18,8171 ITHETAIU,tAWVESan.,M},H2l,NMO ),L = 1,LTH) 
( 6 , 838 } 

(6,617} (THETA(L) , (COERMX(L,H} ,M=l,NNO },L=1,LTH} 
(6,839} 

(6,817} (THETAIL), ( A0C9ER ( L , H ) , 1 , WNO } , L= 1, LTH } 

50 


aOQ FORHAT 

801 FORHAT 

802 FORMAT 

803 FORMAT 

804 FORMAT 


(13A6,A2} 

(lHl,//inX,13AS,A2//} 

(316} 

(lM(i,3X,6HLSMrl =I5,5X,MHWC =I5,5X,5HNNO :I5,5X,4HWE 
(18 ,5F8.T} 


= 15} 


806 FORMAT (14/(10^8,5}} 


807 

FORMAT 

806 

FORMAT 

809 

FORMAT 

AHAU =F6 

BID 

FORMAT 

311 

FORMAT 

813 

FORMAT 

815 

FORMAT 

817 

FORMAT 

331 

FORMAT 

635 

FORMAT 

336 

FORMAT 

837 

FORMAT 

333 

FORMAT 

339 

FORMAT 


END 


(1H0,5MRH0 =I5,bX,7HDELTA 
(3Ib,2F5.3} 

(lHl//,iX,4MMP =I5,5X,4HRF 
,3,5X,6HRETA =Fb.3} 

(812} 

(1H0,4H40 =7114//) 
(1HU,5HTHETA) 

( 1H0,?0X,5HAVRH0// ) 
UX,F8*3,7E14.4} 
(1HO,20X,6H£RSOMX// ) 
(lHU,20X,6HAVERSa// ) 
(1HC,20X,6HVESQMX// ) 
(1HU,2QX,6HAVVESQ// } 
(1HD,20X,6HC0ERMX// } 

( 1H0,2QX,6HADC0ER// } 


=//(lX,lQFlQ.6} } 

=F6.5,6X,BHALPHAF :F6 . 3, 5X , 8R ALP 


U Cl (J 
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SU3R0UmE YATtS 

9,- YATES METHOD SUBROUTIME 

COMHOR KK, Y(16)»BU6) 

II : 2«-*KK 
IIDB2 = II/2 
KKMl : KK-i 
DO 9C9 K=l,KKMI 
DO 9D6 1=1*11*2 
IP102 : (I*n/2 
3IIP102) = YII*ll»vm 
LL = IPID2*IIDB2 
905 BILL) : U I*l)-YII) 

DO 9C7 I=l*II 
907 YU) = BID 
903 CONTINUE 

DO 9C9 1=1*11*2 
IP102 = II^l)/2 
BIIP132) = YII*1)^YII) 

LL = IP1D2^IID32 
BILL) = Y|I»1)-YII) 

909 CONTINUE 
RETURN 
END 
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appendix E 

ILLUSTRATIVE OUTPUT OF COMPUTER PROGRAM POOL9U 


MAY lOt 

1«ID TEM r a.«<UTS*l <*S 

TEN : O.S«l 




»H0 : IS 3ELY» I 







A.lQIt* 1.6AMS2 

.■•JUS .JS2U 

I.A093* 1.221tk 

.2SMS .U<2S 

l.SStkS 

•lOSSS 

•JStSS .I221S 

•7tkOS 

•klTSI 

.S?3l3 

LSItH : <1 MC : 

Ik NYO = 

7 Nt 

: 1030 





Mk I 0 

H3 z 

YF = l.COO 
0 

ALRHAF : 
1 

*500 ALFHAJ 

2 

: .100 
3 

RETA : .830 

5 

S 

5 

THETA 

ayrho 







• 12S 

.2S0 
.SOO 
A.XO 
2. ^0 

.1500.02 

.1500.02 

.isoo.o: 

.1500.C2 

.IEOO.32 

ERSOHX 

.8552*01 

.>rjo*Ci 

•1155*02 

.1510*02 

•1590*02 

.1225*02 

.1302*02 

.1389*02 

.15$6*02 

.1591*02 

.1225*02 

.1253*02 

•1351*02 

.1557*32 

.1589*02 

.1195*02 
.1255.02 
.1352. ■'2 

.1550. "2 

.1599*32 

.1155.02 

.1235*02 

.1333*02 

..537*02 

..589*32 

.ll99*li 
.1225. It 
. 1 325*it 
.193*. 

.12S 

.2S0 

.SOO 

1. uoo 

2. uQ0 

.10(5.01 

.1065.01 

.1056.01 

.1055.01 

.1065.01 

AVERSO 

.9285*00 

.1351*01 

.2851*01 

.2957*01 

•1118*01 

. 1051*01 
.1058*01 
.1068*01 
. 1358*01 
. 1055*01 

. 1038*01 
.1051*01 
.1075*31 
.1058*01 
. 1052*01 

.1335."1 

.1"55."1 

.1373.11 

•1051*01 

.1151*01 

..025*31 

.*153*01 

..057*11 

.1155*01 

.lOss*-;! 

. i:25.i. 
. 13*7. 

. 1:71. 

..353.'-; 

.l'*9.-. 

.12S 
.2S0 
• SOO 
1.300 
2.U00 

•1006.01 

.1005.01 

.1035.01 

.1005.01 

.1005.01 

YES3NX 

.8985*00 

.1009*01 

.1239*01 

.1251*01 

.1022*01 

.9820*00 
. 1033*01 
.1009*31 
. 1035*01 
.1333*31 

.9935*00 

.9985*00 

.1005*31 

.1003*01 

.13C1*31 

.9937*00 

.»955*00 

.1032*01 

.1001*01 

.9979*00 

.*882*03 

.9953*03 

.1031*11 

.1003*01 

.*953*30 

.9150*-, 

, 9919. 
..952.^4, 
,*9o?."'l 
. *93*. 

• 12S 
.250 
.500 
1.000 
2.000 

.2550.01 

•2550*01 

.2550.01 

.2550*01 

.2550*01 

•YYiSa 

.1878*01 

•2735*01 

•1251*02 

.5955*02 

•2255*02 

.2578*31 

.2555*01 

.2530*01 

.2523*01 

.2518*31 

.2555*01 

*2571*01 

•2595*01 

.2538*01 

.2531*01 

.2552*31 

.25T3*ni 

.2555*01 

.2522*11 

.2519*01 

.2595*01 

.2*20*01 

.2595*31 

•2559*01 

•fc9o5*?l 

.25.1*1. 
.:-577.'. 
.2*92.1. 
..977»'i 
. .9 7«. 1. 

.12S 

.2S0 

.500 

l.COO 

2.000 

.2055*01 

.2055*01 

.2055*01 

.2055*01 

.2055*01 

COERMI 

•1552*01 

.1950*01 

.3205*01 

.7552*01 

.3533*01 

. 1959*01 
.2032*31 
.2082*01 
.2352*01 
.2050*31 

.2015*01 

.2017*01 

•2058*01 

*2037*01 

*2032*01 

.1997*''l 

•2037.01 

.2053*01 

..031*01 

.23.9*01 

. 1989*01 
.2102*11 
.2038*01 
. *025*01 
..115*11 

. .977.11 
. 1997. ■'1 
..331*01 

.2019.11 

*23i,7*'i 

.125 

.250 

.500 

1.000 

2.000 

.1250*01 

.5130*01 

.2055*01 

.1033*01 

.5153*00 

ADCOER 

.7709*01 

.5557*01 

.3375*01 

.1717*01 

•5288*00 

.8151*01 
.5115*31 
.2057*01 
. 1033*31 
.5150*30 

.8151*01 

.5101*01 

.2079*01 

•1029*01 

.5153*00 

.1153*01 

.9107*01 

.2071*01 

•1033*01 

.5151*00 

•8103*01 

.9099*31 

.2055*11 

.1028*01 

.3135*10 

.5393* .* 
. 9 39?*'-. 
.2059*1; 
. n:5*"i 
.al.t* •- 

• 125 

• 2S0 
.SOO 

1.000 

2.000 

•3305*02 
•1552*02 
.1250*01 
• 5130*01 
.2055*01 

.3178*02 

.1*25*02 

.1392*02 

.7078*01 

.2180*01 

. 3552*02 
. 1755*02 
. 8770*01 
.5385*31 
.2189*31 

.3553*02 
. 1787*02 
•9039*01 
.5585*01 
.2256*01 

.3553*02 

.1835*02 

.9259*01 

.9505*01 

.2309*01 

*3713*32 

.1878*02 

. 9559.11 

•97i9.01 

•2359.31 

. I795*"2 
. *921*''. 
.97C,3*". 
, 9)19*1. 
*291,’*1. 
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Table 1. - Expressions for degrees of freedoa, suns of squares and test statistics. 









Table 2. 

~ Values 

of j 

"0 

0 

1 

0.0 

2 

2 

0.4 

2 

2 

0.5 

2 

2 

0.7 

2 

2 

0.9 

2 

2 

1.0 

2 

2 

2.0 

2 

3 

4.0 

2 

5 

8.0 

2 

9 

16.0 

2 

n. t 


which 

transfer 

from F 

2 

3 

4 

2 

2 

2 

2 

2 

2 

2 

2 

3 

2 

3 

3 

2 

3 

4 

3 

4 

5 

5 

7 

9 

9 

13 

n.t. 

n.t. 

n.t. 

n.t. 

n.t. 

n.t. 

n.t. 


Uj distribution occurs. 
5 6 

2 2 

3 3 

3 4 

4 5 

5 6 

6 7 

11 13 

n.t. n.t. 

n.t. n.t. 

n.t. n.t. 


n.t 


No transfer occurs 


66k* 62^ ■ 0 


Table 3. - Parameter combinations, Bk +2 * 


k 

«k 

e 

0.125 

0.250 

0.500 

1.000 

2.000 

1 

2.1082 

0.2635 

0.5270 

1.0541 

2.1082 

4.2164 

2 

1.6645 

.2081 

.4161 

.8322 

1.6645 

3.3290 

3 

1.4094 

.1762 

.3524 

.7047 

1.4094 

2.8188 

4 

1.2219 

.1527 

.3055 

.6110 

1.2219 

2.4438 

5 

1.0694 

.1337 

.2674 

.5347 

1.0694 

2.1388 

6 

0.9386 

0.1173 

0.2346 

0.4693 

0.9386 

1.8772 

7 

.8221 

.1028 

.2055 

.4110 

.8221 

1.6442 

8 

.7161 

.0895 

.1790 

.3580 

.7161 

1.4322 

9 

.6176 

.0772 

.1544 

.3088 

.6176 

1.2352 

10 

.5250 

.0656 

.1312 

.2625 

.5250 

1.0500 

11 

0.4368 

0.0546 

0.1092 

0.2184 

0.4368 

0.8736 

12 

.3521 

.0440 

.0880 

.1760 

.3521 

.7042 

13 

.2699 

.0337 

.0675 

.1350 

.2699 

.5398 

14 

.1892 

.0237 

.0473 

.0946 

.1892 

.3784 

15 

.1084 

.0136 

.0271 

.0542 

.1084 

.2168 



1.556 

3.112 

6.225 

12.449 

24.898 


(IBk)’^: 


0.643 


0.321 0.161 0.080 0.040 







Table 4. •* Admissible strategies and their operating characteristics 


(a) nQ - 0. 


"V 


“f 

“u 

m 

1 

> 

V(e2) 

' 'max 

Cae 

,mx 

R 

0.125 

2.0 

0.125 


0.125 

2.0 

0.125 

2.0 

0 


1.0 

1.0 

0.0 

15.00 

15.00 

2.460 

2.460 

33.04 

2.065 

1.1215 

1.0000 

1 

— 

1.0 

.75 

.10 

14.98 

14.98 

2.462 

2.605 

33.02 

2.072 

1.1208 

1.0034 

1 

— 

1.0 

.75 

.20 

14.92 

14.91 

2.453 

2.504 

33.00 

2.084 

1.1202 

1.0092 

1 

— 

1.0 

.75 

.40 

14.72 

14.66 

2.450 

2.585 

32.99 

2.133 

1.1198 

1.0329 

1 


1.0 

.50 

.15 

14.72 

14.55 

2.456 

2.791 

32.97 

2.170 

1.1191 

1.0508 

1 


1.0 

.50 

.20 

14.64 

14.49 

2.443 

2.812 

32.96 

2.181 

1.1188 

1.0562^ 

1 

— 

1.0 

.50 

.25 

14.39 

14.17 

2.442 

2.975 

32.95 

2.240 

1.1185* 

1.0847* 

1 

— 

1.0 

.75 

.54 

14.43 

14.36 

2.450 

7.879 

32.94 

2.375 

1.1181 

1.1501 

1 

— 

1.0 

.75 

.55 

14.42 

14.36 

2.454 

8.715 

32.92 

2.404 

1.1174 

1.1642 

1 

— 

1.0 

.50 

.35 

14.04 

13.75 

2.443 

8.117 

32.90 

2.808 

1.1168 

1.3598 

1 


1.0 

.50 

.40 

13.96 

13.60 

2.422 

13.75 

32.88 

3.178 

1.1161 

1.5390 

1 

— 

1.0 

.50 

.45 

13.78 

13.34 

2.398 

34.87 

32.70 

3.910 

1.1100 

1.8935 

1 

— 

1.0 

.50 

.50 

13.36 

12.89 

2.458 

98.42 

32.58 

4.931 

1.1059 

2.3879 

1 

— 

1.0 

.50 

.55 

13.26 

12.79 

2.429 

167.5 

32.55 

5.479 

1.1049 

2.6533 

1 

— 

1.0 

.50 

.75 

12.57 

11.88 

2.383 

2135.0 

32.46 

10.00 

1.1018 

4.8426 

1 

— 

1.0 

.50 

.80 

12.50 

11.74 

2.409 

3159.0 

32.40 

11.11 

1.1000 

5.3801 

1 

— 

1.0 

.50 

.85 

12.32 

11.57 

2.299 

4735.0 

32.28 

12.23 

1.0957 

5.9225 

1 

— 

l.O 

.50 

.90 

12.20 

11.36 

2.270 

9239.0 

32.27 

14.23 

1.0954 

6.8910 

1 

— 

1.0 

.50 

.95 

12.05 

11.13 

2.253 

18610.0 

32.03 

16.87 

1.0872 

8.1695 

1 

-- 

1.0 

.05 

.70 

6.029 

5.710 

1.568 

770.8 

30.53 

19.05 

1.0363 

9.2252 

1 

— 

1.0 

.10 

.80 

6.209 


1.631 

2632.0 

30.39 

22.36 

1.0316 

10.8281 

1 

— 

1.0 

.05 

.75 

5.137 

■SEE 

1.472 

1581.0 

29.72 

23.17 

1.0088 

11.2203 

1 

— 

1.0 

.025 

.75 

4.488 

4.389 

1.465 

903.1 

29.61 

23.63 

1.0051 

11.4431 

5 

“““ 

1.0 

.05 

.75 

4.291 

4.000 

1.497 

172.6 

29.46 

24.09 

1.000 

11.6659 


A 

Security regret strategy. 






















Table 4. - Cont’d 


(b) no - 1. 


®p 


®F 

“u 


7 


^ae 

.nx 

R 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0.125 

msm 

0 

2.0 

0.50 

0.75 

0.65 

10.66 

14.90 

2.291 

2.512 

33.05 

2.118 

1.0875 

1.0000 

0 

3.0 

.50 

.25 

.80 

8.747 

14.91 

1.898 

2.508 

32.04 

2.119 

1.0543 

1.0005 

0 

3.0 

.50 

.10 

.80 

8.442 

14.90 

1.878 

22.46 

31.78 

2.180 

1.0457* 

1.0293* 

0 

2.0 

.50 

.25 

.75 

7.463 

14.65 

1.714 

408.0 

31.61 

4.094 

1.0401 

1.9330 

0 

2.0 

.50 

.10 

.80 

6.741 

14.38 

1.692 

1029.0 

31.20 

5.989 

1.0267 

2.8277 

0 

0.0 

.50 

.10 

.75 

5.780 

12.27 

1.581 

3796.0 

30.82 

12.17 

1.0141 

5.7460 

0 

0.5 

.25 

.05 

.80 

5.209 

10.60 

1.487 

5046.0 

30.60 

15.57 

1.0069 

7.3513 

0 

4.0 

.01 

.05 

.75 

4.305 

4.737 

1.551 

1607.0 

30.53 

23.90 

1.0046 

11.2842 

o 

4.0 

.01 

.025 

.75 

4.194 

4.657 

1.536 

1426.0 

30.51 

24.02 

1.0039 

11.3409 

0 

8.0 

.01 

.05 

.75 

4.281 

4.649 

1.536 

1408.0 

30.47 

24.03 

1.0026 

11.3456 

0 

4.0 

.005 

.05 

.75 

4.275 

4.404 

1.539 

1024.0 

30.46 

24.30 

1.0023 

11.4731 

0 

4.0 

.005 

.025 

.75 

4.164 

4.316 

1.524 

811.3 

30.44 

24.43 

1.0016 

11.5345 

0 

8.0 

.005 

.05 

.775 

4.209 

4.276 

1.517 

716.3 

30.39 

24.50 

1.0000 

11.5675 


*Security regret strategy. 


































Table 4. - Cont'd. 


(c) no * 2. 



1 

®F 


i 

P 

V(e 

2) 

'max 

^ae 

•nx 

R 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0 

2.0 

0.75 

0.50 

0.90 

13.12 

14.98 

2.437 

2.519 

34.40 

2.187 

1.1058 

1.0000 

0 

1.0 

.50 

.75 

.70 

10.10 

14.88 

2.340 

2.533 

33.98 

2.188 

1.0923 

1.0005 

0 

2.0 

.50 

.25 

.85 

9.957 

14.91 

2.048 

2.518 

33.20 

2.189 

1.0672 

1.0009 

0 

3.0 

.25 

.50 

.85 

8.959 

14.42 

2.227 

7.804 

33.14 

2.252 

1.0653* 

1.0297* 

0 

1.0 

.50 

.50 

.74 

8.510 

14.83 

2.180 

31.24 

33.08 

2.466 

1.0633 

1.1276 

0 

1.0 

.50 

.50 

.80 

8.142 

14.81 

2.157 

65.72 

32.93 

2.610 

1.0585 

1.1934 

0 

0.0 

.75 

.50 

.80 

8.045 

14.61 

2.157 

142.3 

32.91 

3.152 

1.0579 

1.4412 

0 

2.0 

.25 

.25 

.75 

7.508 

14.28 

1.922 

384.4 

32.70 

4.022 

1.0511 

1.8390 

0 

2.0 

.25 

.25 

.80 

7.327 

14.28 

1.895 

416.8 

32.57 

4.096 

1.0469 

1.8729 

0 

1.0 

.50 

.10 

.80 

6.492 

14.43 

1.695 

960.4 

32.04 

5.921 

1.0299 

2.7074 

0 

1.0 

.50 

.025 

.75 

6.361 

14.30 

1.674 

1221.0 

31.96 

6.638 

1.0273 

3.0352 

0 

0.0 

.50 

.10 

.80 

5.362 

12.30 

1.580 

3680.0 

31.72 

12.35 

1.0196 

5.6470 

0 

0.0 

.50 

.05 

.80 

5.206 

12.04 

1.546 

3958.0 

31.66 

13.04 

1.0177 

5.9625 

0 

0.0 

.50 

.025 

.75 

5.136 

11.91 

1.541 

4106.0 

31.59 

13.38 

1.0154 

6.1180 

0 

1.0 

.25 

.025 

.80 

5.176 

11.24 

1.533 

4676.0 

31.57 

14.85 

1.0148 

6.7901 

0 

0.0 

.25 

.05 

.80 

5.045 

10.46 

1.524 

5018.0 

31.56 

16.18 

1.0145 

7.3983 

0 

0.0 

.25 

.025 

.75 

4.976 

10.30 

1.517 

5086.0 

31.48 

16.51 

1.0119 

7.5492 

0 

0.0 

.10 

.025 

.75 

4.909 

9.569 

1.516 

5249.0 

31.42 

17.77 

1.0100 

8.1253 

0 

0.0 

.10 

.01 

.75 

4.840 

9.521 

1.496 

5262.0 

31.37 

17.87 

1.0084 

8.1710 

0 

0.0 

.10 

.01 

.80 

4.829 

9.521 

1.499 

5262.0 

31.37 

17.87 

1.0084 

8.1710 

0 

0.0 

.025 

.025 

.75 

4.864 

9.292 

1.517 

5264.0 

31.34 

18.22 

1.0074 

8.3310 

0 

0.0 

.025 

.01 

.75 

4.795 

9.244 

1.497 

5271.0 

31.29 

18.32 

1.0058 

8.3768 

0 

0.0 

.025 

.01 

.80 

4.784 

9.244 

1.500 

5271.0 

31.29 

18.32 

1.0058 

8.3768 

0 

2.0 

.002 

.10 

.80 

4.362 

4.436 

1.491 

1173.0 

31.26 

24.88 

1.0048 

11.3763 

0 

2.0 

.005 

.05 

.75 

4.276 

4.349 

1.480 

915.8 

31.24 

25.06 

1.0042 

11.4586 

0 

2.0 

.002 

.05 

.75 

4.245 

4.240 

1.468 

707.4 

31.12 

25.20 

1.0003 

11.5226 

0 

4.0 

.002 

.05 

.80 

4.086 

4.113 

1.478 

394.2 

31.11 

25.41 

l.OOOO 

11.6187 


*- 

Security regret strategy. 
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Table 4 


Cont*d 


(d) Hg • 3. 


Bp 

1 

®F 

“u 

\ 

7 

V(e 

2) 

'max 

^ae 

»mx 

K 

r\e 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

.. . 

0 

1.0 

0.75 

0.50 


10.53 

14.97 

2.119 

2.523 

34.41 

2.244 

1.0770 

1.0000 

0 

1.0 

.50 

.50 

.80 

7.456 

14.89 

1.888 

2.532 

33.63 

2.246 

1.0526 

1.0009 

0 

0.0 

.75 

.50 


6.581 

14.73 

1.691 

11.95 

33.29 

2.301 

1.0419* 

1.0254* 

0 

0.7 

.50 

.50 


6.584 

14.83 

1.665 

30.94 

33.20 

2.356 

1.0391 

1.0499 

0 

1.0 

.50 

.10 

.80 

6.418 

14.86 

1.706 

64.86 

32.94 

2.578 

1.0310 

1.1488 

0 

1.0 

.50 

.05 


6.372 

14.86 

1.705 

78.33 

32.87 

2.684 

1.0289 

1.1961 

0 

0.7 

.50 

.10 

.75 

5.195 

14.44 

1.520 

833.1 

32.39 

5.725 

1.0138 

2.5512 

0 

0.7 

.50 

.025 

.75 

4.988 

14.28 

1.470 

1161.0 

32.21 

6.689 

1.0081 

2.9808 

0 

0.0 

.50 

.025 

.75 

4.387 

12.31 

1.491 

3665.0 

32.19 

12.63 

1.0075 

5.6233 

0 

0.7 

.25 

.05 

.80 

4.354 

12.24 

1.483 

3765.0 

32.13 

12.79 

1.0056 

5.6996 

0 

0.7 

.25 

.025 

.75 

4.360 

12.08 

1.475 

3957.0 

32.09 

13.26 

1.0044 

5.9091 

0 

0.0 

.25 

.025 

.75 

4.298 

11.12 

1.474 

4662.0 

32.06 

15.20 

1.0034 

6.7736 

0 

1.0 

.10 

.025 

.75 

4.281 

9.967 

1.463 

5162.0 

32.02 

17.16 

1.0022 

7.6471 

0 

2.0 

.001 

.05 


Km 

4.117 

1.486 

427.6 

31.97 

26.08 

1.0006 

11.6221 

0 

2.0 

.001 

.025 

.75 

Bb 

4.097 

1.477 

371.7 

31.95 

26.12 

1.0000 

11.6399 


* 


Security regret strategy. 















Table 4. - Cont'd 


(e) « 4. 


nip 


“f 

“u 

r \9 

A 

■ 

3 

V(e 

2) 

■'max 

'ae 

•mx 

R 

0.125 

1.0 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0 

0.5 

0.75 

0.50 

0.85 

7.038 

14.96 

1.845 

2.501 

34.38 

2.300 

1.0536 

1.0000 

0 

1.0 

.50 

.05 

.80 

6.992 

14.89 

1.864 

2.519 

33.93 

2.304 

1.0398 

1.0017 

0 

0.5 

.50 

.50 

.80 

5.904 

14.83 

1.605 

2.533 

33.89 

2.309 

1.0386* 

1.0039* 

0 

0.0 

.50 

.50 

.80 

5.616 

14.48 

1.591 

12.77 

33.71 

2.421 

1.0331 

1.0526 

0 

1.0 

.25 

.10 

.80 

4.830 

14.48 

1.639 

151.2 

33.12 

3.096 

1.0150 

1.3461 

0 

0.0 

.50 

.25 

.80 

4.723 

14.04 

1.539 

588.4 

33.07 

5.257 

1.0135 

2.2857 

0 

0.5 

.50 

.05 

.75 

4.737 

14.44 

1.486 

854.5 

32.94 

5.930 

1.0095 

2.5783 

0 

0.9 

.25 

.05 

.80 

4.394 

14.03 

1.484 

1131.0 

32.90 

6.710 

1.0083 

2.9174 

0 

0.0 

.50 

.05 

.80 

4.241 

13.01 

1.465 

2700.0 

32.74 

10.69 

1.0034 

4.6478 

0 

0.5 

.25 

.05 

.80 

4.243 

12.85 

1.465 

2923.0 

32.67 

11.15 

1.0012 

4.8478 

0 

0.0 

.25 

.05 

.80 

4.186 

12.13 

1.460 

3653.0 

32.64 

12.86 

1.0003 

5.5913 

0 

0.9 

— 

.10 

.05 

.80 

4.142 

— 

11.01 

1.451 

4547.0 

32.63 

15.11 

1.0000 

6.5696 


*Security regret strategy. 





























Table 4 


Cont'd 


(f) Hq - 5. 


fflp 


®F 

“u 

HI 

7^ 

V(e2) 

' 'max 

C 

ae 

,mx 

R 

Eg 

0.125 

2.0 

0 . 1:5 

2.0 

0.125 

2.0 

0.125 

2.0 

0 

0.6 

0.50 

0.10 

0.60 

7.120 

14.89 

1.950 

2.488 

35.92 

2.352 

1.0719 

1.0000 

0 

0.7 

.50 

.50 

.80 

6.049 

14.89 

1.739 

2.486 

34.81 

2.354 

1.0388 

1.0009 

0 

0.7 

.50 

.10 

.80 

5.233 

14.88 

1.616 


34.30 

2.360 

1.0236 

1.0034* 

0 

1.0 

.25 

.10 

.80 

4.834 

14.53 

1.615 

2.567 

33.97 

2.394 

1.0137 

1.0179 

0 

0.8 

.25 

.25 

.80 

4.581 

14.44 

1.537 

88.61 

33.96 

2.925 

1.0134 

1.2436 

0 

0.5 

.50 

.10 

.75 

4.616 

14.68 

1.508 


33.90 

3.826 

1.0116 

1.6267 

0 

0.5 

.50 

.10 

.80 

4.489 

14.68 

1.487 


33.71 

3.854 

1.0060 

1.6386 

0 

0.7 

.25 

.10 

.80 

4.247 

14.30 

1.492 

468.3 

33.70 

4.703 

1.0057 

1.9996 

0 

0.5 

.50 

.05 

.80 

4.403 

14.55 

1.462 

571.1 

33.60 

5.120 

1.0027 

2.1769 

0 

0.5 

.50 

.025 

.80 

4.388 

14.51 

1.462 

691.4 

33.58 

5.540 

1.0021 

2.3554 

0 

0.5 

.50 

.01 

.80 

4.379 

14.47 

1.456 

805.2 

33.57 

5.889 

1.0018 

2.5038 

0 

0.5 

.25 

.05 

.80 

4.101 

13.27 

1.466 

2202.0 

33.51 

9.744 

1.0000 

4.1429 


Security regret strategy. 
























Table 4. - Concluded. 

(g) Hq » 6. 


nip 

■ 

“f 

a 

U 


P 

V ( e 2) 

' -'max 

"ae 

,mx 

R 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0.125 

2.0 

0 

0.6 

0.50 

0.10 

0.60 

6.862 

14.89 

1.805 

2.479 

36.77 

2.401 

1.0736 

1.0000 

0 

0.7 

.50 

.10 

.80 

6.071 

14.89 

1.786 

2.478 

35.38 

2.402 

1.0330 

1.0004 

0 

0.5 

.50 

.05 

.80 

4.850 

14.88 

1.549 

2.485 

34.72 

2.408 

1.0137* 

1.0029* 

0 

0.9 

.25 

.05 

.80 

4.548 

14.55 

1.679 

2.536 

34.66 

2.437 

1.0120 

1.0150 

0 

0.7 

.25 

.10 

.80 

4.336 

14.54 

1.500 

30.30 

34.55 

2.524 

1.0088 

1.0512 

0 

1.0 

.10 

.10 

.80 

4.239 

13.75 

1.499 

97.63 

34.48 

3.274 

1.0067 

1.3636 

0 

0.5 

.25 

.10 

.80 

4.159 

14.41 

1.496 

245.1 

34.43 

3.824 

1.0053 

1.5927 

0 


.25 

.05 

.80 

4.080 


1.493 

461.5 

34.34 

4.754 

1.0026 

1.9800 

0 

0.4 

.50 

.025 

.75 

4.324 

14.62 

1.453 

476.4 

34.30 

4.804 

1.0015 

2.0008 

0 


.50 

.025 

.80 

4.037 

13.65 

1.477 

1573.0 

34.25 

8.384 

1.0000 

3.4919 


Security regret strategy. 
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Table 5. - Values of p as functions of Hq, 0, and choice of strategy. 



Strategy 

^Minimum 

Cae.mx(2-0) 

^Security Regret 

‘Minimum 


0 

”0 

0.125 

2.0 

0.125 

■n 

0.125 

2.0 

0 

15.00 

15.00 

14.39 

14.17 

4.291 

4.000 

■ 

10.66 

14.90 

8.442 

14.90 

4.209 

4.276 

B 

13.12 

14.98 

8.959 

14.42 

4.086 

4.113 

B 

10.53 

14.97 

6.581 

14.73 

4.075 

4.097 

B 

7.038 

14.96 

5.904 

14.83 

4.142 

11.01 

B 

7.120 

14.89 

4.834 

14.53 

4.101 

13.27 

B 

6.862 

14.89 

4.850 

14.88 

4.037 

13.65 


®From first row of table 4. 

^From asterisked results in table 4. 
CFrom last row of table 4. 












GENERATE RNIII 
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Y0BS(II-YMU(I.U-»RN(II 

1*1, NC 

224 

CAU YATES TO GENERATE 
Bil) FROM YOBS(I) 


3. - CONSTRUCTION AND ORDERING OF 
MAN SQUARES 
2(11 • BIHII* Bd^lllFNC 
BFMIII • BdVFNC 
309 I • 1. NC 
313 ORDER Zdl 



6. - ACCUMULATION OF ERRORS 

ERROR^ • (B(NCPIM1)-YMU(I.I»*»2 


|m - m*i| 

68 ^ 

<^>NN^>^ 

690 

1 1 - 1*1 | — < 


'L>m>^ 


^N*Ntl]--^<:;N>N^^ 


Figure 1. - Uttafirriys YOBSll), Bdl, and BFMdI. 
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la) Computation o( sums ot squaras. 

Figure 2 - Flow chart ot program P0019U. Numtwrs In I ) art equation 
numbers of text. Three digit integers are statement numbers In 
Appendix D. 















Id DrlFlion Ot Insiqnlllcant corlllcientt. 
Figure 2. - CorKluded. 
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